My side, your side and the evidence: Discovering aligned actor groups and the narratives they weave
Year: 2023

Abstract:
News reports about emerging issues often include several conflicting story lines. Individual stories can be conceptualized as samples from an underlying mixture of competing narratives. The automated identification of these distinct narratives from unstructured text is a fundamental yet difficult task in Computational Linguistics since narratives are often intertwined and only implicitly conveyed in text. In this paper, we consider a more feasible proxy task: Identify the distinct sets of aligned story actors responsible for sustaining the issue-specific narratives. Discovering aligned actors, and the groups these alignments create, brings us closer to estimating the narrative that each group represents. With the help of Large Language Models (LLM), we address this task by: (i) Introducing a corpus of text segments rich in narrative content associated with six different current issues; (ii) Introducing a novel two-step graph-based framework that (a) identifies alignments between actors (INCANT) and (b) extracts aligned actor groups using the network structure (TAMPA). Amazon Mechanical Turk evaluations demonstrate the effectiveness of our framework. Across domains, alignment relationships from INCANT are accurate (macro F1 ≥ 0.75) and actor groups from TAMPA are preferred over 2 non-trivial baseline models (ACC ≥ 0.75). 1 Background and Motivation Discussions about current events in public forums involve consensus building, with the exchange of beliefs and perspectives producing competing, often conflicting, narratives. A person reading these discussions parses natural language and is able to tease out and maintain representations of the various narratives, including the central actors, their alignments, and the often-contrasting points-ofview presented by the narratives. Replicating this type of comprehension in machines by creating interpretable, mathematical representations of narrative structure is a field of continued computational linguistics efforts (Bailey, 1999; Beatty, 2016). A narrative is usually modeled as a narrative network of actors (nodes) and their inter-actor relationships (edges). This graph building is, however, a challenging aggregation task since the same narrative can be expressed in natural language in several ways. Conversely, a given text span can include signatures of several underlying narratives. It is worth noting that a coherent narrative usually features a small set of critical actors that emerge through the give and take of online discussions and provides a distilled representation of a particular world view. We refer to these key sets of critical actors that are narratively aligned to a shared worldview as “actor groups”. People reading or participating in the discussion, in turn, support or even identify with these story actor groups, ensuring the persistence of the narrative in the discussion domain. Identifying these groups of aligned actors is essential to defining the boundaries of a narrative, its current scope and, possibly, its future viability (i.e. if people do not recognize actor groups as central to a narrative, that group and its constitutive members is likely to disappear over time from the narrative space). We therefore consider the detection of actor groups from text as an accessible first step in the larger task of estimating the total narrative structure. Task: Discovering actor groups from text Given a corpus of domain-specific freeform text, construct a model to discover the actor groups that undergird the disparate narratives in that domain. The task of identifying actor groups adds to a growing body of computational linguistics work that identifies salient features of the abstracted narrative structure by exploiting the subtle contextual clues available in free-form text: for instance, In-

heading: 1 Background and Motivation
text: Discussions about current events in public forums involve consensus building, with the exchange of beliefs and perspectives producing competing, often conflicting, narratives. A person reading these discussions parses natural language and is able to tease out and maintain representations of the various narratives, including the central actors, their alignments, and the often-contrasting points-ofview presented by the narratives. Replicating this type of comprehension in machines by creating interpretable, mathematical representations of narra-
tive structure is a field of continued computational linguistics efforts (Bailey, 1999; Beatty, 2016). A narrative is usually modeled as a narrative network of actors (nodes) and their inter-actor relationships (edges). This graph building is, however, a challenging aggregation task since the same narrative can be expressed in natural language in several ways. Conversely, a given text span can include signatures of several underlying narratives.
It is worth noting that a coherent narrative usually features a small set of critical actors that emerge through the give and take of online discussions and provides a distilled representation of a particular world view. We refer to these key sets of critical actors that are narratively aligned to a shared worldview as “actor groups”. People reading or participating in the discussion, in turn, support or even identify with these story actor groups, ensuring the persistence of the narrative in the discussion domain. Identifying these groups of aligned actors is essential to defining the boundaries of a narrative, its current scope and, possibly, its future viability (i.e. if people do not recognize actor groups as central to a narrative, that group and its constitutive members is likely to disappear over time from the narrative space). We therefore consider the detection of actor groups from text as an accessible first step in the larger task of estimating the total narrative structure.
Task: Discovering actor groups from text
Given a corpus of domain-specific freeform text, construct a model to discover the actor groups that undergird the disparate narratives in that domain.
The task of identifying actor groups adds to a growing body of computational linguistics work that identifies salient features of the abstracted narrative structure by exploiting the subtle contextual clues available in free-form text: for instance, In-
8938
siders and Outsiders (Holur et al., 2022), Conspiratorial Actors (Shahsavari et al., 2020b), Supernodes and Contextual Groups (Tangherlini et al., 2020), and inter-actor event sequencing (Shahsavari et al., 2020a; Holur et al., 2021) (see Related Works Sec. 2 for an extended discussion). Discovering aligned actors as a means to construct actor groups: A set of mutually aligned actors forms an actor group. Alignment is subtly implied via the inter-actor relationships – often a VERB phrase – in free-range text: Consider, for example, in the news domain of Gun Regulations in the United States, a text segment:
{Republicans} −→ are funded by −→ the {NRA} suggests {Republicans, NRA} are aligned. In contrast, another segment,
{Democrats} −→ laid out their anti- −→ {Second Amendment} credentials
implies that “Democrats” are opposed to the “Second Amendment” and the two actors {Democrats, Second Amendment} are disaligned. Tasking a model to discover alignment relationships, a process that comes quite naturally to humans, presents two distinct computational challenges: Prob-1 Understanding alignment requires human experience: The context traces in language imply but do not explicitly state the alignment between a pair of actors. From the sample text concerning Gun Regulations, we observe that the {NRA and Republicans} were aligned because the NRA funded the Republican party; it is widely accepted in American politics that funding signals support. In another text span, Democrats → encourage → Black women and men to vote indicates {Democrats}, {Black women and men} are aligned because encouragement is a form of validation. These alignments are trivial to a reader – that offering money and emotional support imply alignment; however, the entire set of phrases that convey alignment in natural language is infinite. Finding the means to map these phrases onto a latent alignment dimension is a fundamental challenge.
Prob-2 Alignment is transitive across a narrative network: Alignment between one pair of actors has the capacity to influence the alignment across other actor pairs in a process that echoes the well-known feature of Structural Balance Theory (Cartwright and Harary, 1956; Davis, 1967): a friend of a friend is a friend while an enemy of an enemy is a friend, etc. In the Gun
Regulations domain, the pair of alignment relationships: “Democrats → sought to ban → the “NRA” (disalignment) and “the Republicans → supported → the NRA” (alignment) jointly implies that {Democrats, Republicans} are disaligned, despite the absence of a direct relationship conveying alignment between them. Consider a third disalignment: “the NRA → opposed → a gun safety law”. Since, {NRA, a gun safety law} are disaligned and {NRA, Democrats} are disaligned, according to transitivity, it follows that {Democrats, a gun safety law} are aligned. Therefore, modeling this transitivity requires unifying alignment constraints across disparate contexts and text spans.

heading: 2 Our Approach and Related Work
text: Computational efforts to address Prob-1 model human experience by adapting pre-trained large language models (PLM). These models demonstrate considerable semantic awareness in several well-known NLP tasks (a product of the knowledge embedded in the exhaustive training corpora); Semantic Role Labeling (SRL) (Zhang et al., 2022), Question-Answering (QA) (Liu et al., 2019a), Sentiment Analysis (SA) (Yin et al., 2020), and Language Generation (LG) (Floridi and Chiriatti, 2020) for instance, all make use of pre-training to boost performance.
The transitivity requirement in Prob-2 is often addressed by fine-tuning PLMs on biased datasets containing implicit transitivity constraints (Holur et al., 2022; Liu et al., 2019a). Fine-tuning weights encourages generalization across data samples. However, these fine-tuned models are datasetspecific and must be retrained for every encountered domain: an expensive and time-intensive task. Alternative approaches use models that are trained to generate an external representation of the domain, often in the form of a network (Yu et al., 2022). The network structure enables higher-order consensus insights.
The semantic awareness exhibited by PLMs motivate the adoption of a transfer learning approach to extract alignment implicit in text segments. In our work, this is facilitated by Question-Answering (QA) (see Sec. 2.1) that outputs an alignment network specific to a conversation domain: the network is a joint representation of individual pairwise alignment relationships between actors (INCANT). Actor groups are identified by exploiting this network structure (TAMPA).
The task of discovering aligned actor groups has strong parallels to identifying homophily between users on social media platforms (Khanam et al., 2022). Homophily refers to the tendency for individuals to interact more frequently with those who share similar beliefs and attitudes. Identifying homophilic user groups involves exploiting latent features in the social media with which the users interact; for example, Šćepanović et al. (2017) identifies user cohorts on Twitter by profiling their engagement with political parties; meanwhile Del Tredici et al. (2019) utilizes the neighborhood of a user within a social network to enable inter-user comparison. Our work extends these ethnographic efforts to the narrative landscape: we identify groups of actors that feature in the narrative using the contextual alignment clues present in the language.

heading: 2.1 Alignment modeling using question-answering
text: Recent Natural Language Understanding (NLU) models have implemented a Question-Answering (QA) framework to replicate the iterative process of knowledge acquisition in humans (He et al., 2015; Gutman Music et al., 2022). This framework aims to identify template answer spans that populate a latent knowledge graph, and several network algorithms are applied to infer long-range relationships on this network with multi-hop reasoning and link prediction (Diefenbach et al., 2018; Buck et al., 2017). Similarly, narrative theorists have proposed that questions clarify a narrative’s “fillers” or facets (Bailey, 1999). Therefore, a QA-approach to model alignment, an essential facet of narrative structure, should be effective. Crowdsourcing question templates for align-
ment retrieval: Deciphering alignment relationships requires asking specialized questions: for example, a reader knows that for a person-actor (phrase), we can identify its alignment constraints by asking: whom the {phrase} supports, whom the {phrase} opposes, whom the {phrase} works with, what the {phrase} protects/threatens etc. Typical Question Generation (QG) task setups involve predicting the optimal question given a {context} or {context,answer} tuple (Xiao et al., 2020; Pan et al., 2019); we propose a simple yet effective model to recommend alignment-oriented questions.
Our QG model prioritizes questions conditioned on the NER tag – such as person (PER), or organization (ORG) – of an encountered actor in text. Following an approach similar to He et al. (2015), we crowdsource a basis set of question templates (q) and associated alignment score zq ∈ {−1,−0.25,−0.1,+0.1,+0.25,+1} for each NER tag from N = 5 annotators in the en_US locale. zq = −1 indicates that the {phrase} actor span in q and the resulting answer are disaligned; a score zq = +1 suggests strong alignment. Popular templates (freq > 2) chosen by annotators are presented in Tab. 1) along with the mode alignment score. Transfer learning through Question-Answering: We reorient the comprehension abilities of TransformerQA (Liu et al., 2019b), a RoBERTa-large QA PLM trained on the SQuAD dataset (Rajpurkar et al., 2016), to map free-form text relationships to alignment constraints (see Prob-1). For an encountered actor s in a text segment x, we identify its NER tag (Honnibal et al., 2020) and associated question templates qs. {phrase} is replaced by s to create a coherent question. Typical QA models
learn parameters ϕ such that p(t|s, qs;x, ϕ) is maximized, where t is the correct answer/answer span within x. The set of {subject, question, answer} tuples form the alignment network (see Section 4.1).

heading: 3 Data Collection
text: News reports are a fertile ground for exploring the formation of opposing narratives and their attendant actor groups since, for any domain, these accounts contain fragments of the various emergent perspectives, the actors aligned with each narrative and the contrast between potentially opposing sides. While individual news articles may favor one narrative perspective over another, a large corpus of articles concerning a single event or issue may, in the aggregate, capture a wide range of these conflicting (sub)narratives. We use a bootstrapped weakly-supervised process to assemble a corpus of such articles particular to a domain C: 1. Assemble search terms: A small set of 5− 10 core terms and phrases associated to Ci is manually curated (see Table 2 for domains and seeds);
2. Discover news articles: We use GDELT, a
Google Jigsaw-powered open real-time news indexing service1, that pulls recent news articles that match each search term. The search is limited to the en_US locale and to articles published within the last 90 days. GDELT was scraped on Nov 11, 2022. Returned articles are cleaned and common acronyms are resolved2.
We believe there are sufficient actors who are influential in swaying consensus opinion and are unlikely to change their pairwise alignments during the 3-month window. This stabilizes the performance of the inter-actor alignment framework TAMPA (see Sec. 4.2.1) as indicated by our results. The proposed framework also enables identifying actors that switch sides during the observation time window: such actors are positioned by the framework at the outskirts of the core aligned actor groups enabling us to discover the multitude of groups to which they are aligned. For example, we find that in the Ukraine War domain, many Republicans were aligned to Russia (Fig. 4). However, Mitt Romney, a moderate Republican, aligns weakly with Ukraine.
The 6 evaluated domains in Tab. 2 have significantly different time frames, ranging from longstanding debates such as Roe v. Wade to more recent events like the War in Ukraine. These domains also involve a diverse set of actors, range in scope from national issues like Gun Regulations to global concerns like Recession Fears, and are uni-
1www.gdeltproject.org 2https://aspe.hhs.gov/common-acronyms
versally recognized as contentious, with multiple viewpoints to consider.
Segmenting the long-form text: Transformerbased models accept a limited token length of context. We split the news articles into smaller segments while retaining many long-range coreference dependencies: 1. Auto-regressive coreference resolution: Each news article is sentence-tokenized {s1, s2, . . . , sN}. The auto-regressive seq-2seq module greedily resolves references in a sliding window k = 5 {si, . . . , si+k} using a Transformer model trained on OntoNotes 5.0 (Lee et al., 2018). The enriched sentences {ŝi, ŝ2, . . . , ŝi+k} replace the original set, and the process is repeated after moving the window by a stride s = 2. The updated sequence is {ŝ1, ŝ2, . . . , ŝN}. 2. Segment with overlap: A moving window of length l = 3 and stride d = 2 partitions {ŝ1, ŝ2, . . . , ŝN} into fragmented shorter sequences to retain sufficient contextual information per segment for inference with downstream Transformer models while remaining computationally feasible at scale.
In this way, we construct X , the set of l-segment spans extracted from news articles specific to domain C. Data statistics for the specific Cis evaluated in this work are presented in Table 3.

heading: 4 Methods

heading: 4.1 INCANT: The INter-aCtor Alignment NeTwork
text: We estimate the inter-actor alignment network G(V,E) by identifying the set of relationship tuples R that comprise G. Recall from Section 2.1 that every relationship r ∈ R is of the form {s, qs, t} The INCANT network estimation process f parameterized by θ estimates the likelihood
of each alignment relationship r := {s, qs, t} given a text segment x:
p(s,qs, t|x, θ) = p(t|s, qs;x, θ)︸ ︷︷ ︸
C
p(qs|s;x, θ)︸ ︷︷ ︸ B p(s|x, θ)︸ ︷︷ ︸ A . (1)
{A} p(s|x, θ): the likelihood of choosing node (actor) s from x. Named Entities (NE) present in x are eligible source nodes and equally likely;
{B} p(qs|s;x, θ) := p(qs|NER(s);x, θ): the likelihood of choosing a question template qs from source node s to potential target t: recall that a question template’s eligibility is conditioned on the NER tag of s;
{C} p(t|s, qs;x, θ): the standard QuestionAnswering (QA) inference task covered in Section 2.1.
For a given text span x, let the set of potential alignment relationships be Φx. |Φx| = NE(x)× |Q| where NE(·) is the set of named entities in x (representing the set of potential source nodes) and Q is the set of all question templates. fθ assigns a likelihood score to each relationship in Φx. Those relationships whose likelihood exceeds a threshold λ (= 0.7) are eligible for constructing the alignment network Gx; the aggregated domain-specific alignment network G = ∪x∈XGx. G is a signed, multi-edge, directed alignment network. Note that target actors, as opposed to source actors, need not be named entities. We apply processing steps to G prior to actor group identification: (a) The alignment score zq corresponding to each question qs is used as the edge weight (see Tab. 1); (b) Multiple directed edges between a node pair are collapsed into a single undirected edge and the edge weights are averaged; (c) Actors with sparse connectivity (degree = 1) are ignored; and (d) The GCC of G is used for further evaluation. Steps (c) and (d) together help to highlight the alignment subnetwork that features the most prominent narratives in the domain. The resulting weighted graph is termed the INCANT network particular to domain C and denoted by Ĝ. INCANT network relationships are evaluated using Amazon Mechnical Turk (AMT) (see Tab. 4 for results and Appendix Sec. B for instructions).

heading: 4.2 From INCANT networks to actor groups
text: Given an INCANT network Ĝ(V̂ , Ê), we identify the actor groups that constitute the distinct narratives. This task is represented by a partitioning of
Ĝ: we identify fmap : V̂ → C, where c ∈ C is a subset of actors c ⊂ V̂ . The narrative subnetwork for c contains those edges est ∈ Ê where {s, t} ∈ c.

heading: 4.2.1 TAMPA: Transitive Alignment via Message PAssing
text: We describe a framework to construct numerical actor representations that can be compared using a distance measure and clustered. In the context of an INCANT network Ĝ, the actor representation learning task translates to learning node embeddings. We denote the embedding for node s by hs ∈ RD (D = 3). These embeddings are tuned such that our distance measure, the cosine distance, d(s, t) = 1 − hs·ht||hs||||ht|| , d ∈ [0, 2], is small for aligned actors, and large for opposing ones. Solve:
min h1,...,h|V̂ |
∑
{v1,v2}∈V̂ d(v1, v2)× w(v1, v2), (2)
where wv1,v2 is the alignment score between v1, v2. Solving Eq. 2 directly is intractable since we do not have the alignment constraint between every pair of actors: Ĝ is data-driven and not fullyconnected. To overcome this, we describe a message passing approach to inferring alignment implicit in the network structure. Similar message passing approaches have been shown to be successful in refining node embeddings in the context of co-occurrence networks derived from text (Pujari and Goldwasser, 2021).
Learning graph node embeddings using message passing: The transitive nature of alignment (see Prob-2) allows us to define an effective alignment score z̃v1,v2 between any pair of actors {v1, v2} by considering a random walk (Bondy et al., 1976) from v1 to v2: for a walk of length L, {v1, t1, t2, . . . , v2},
z̃v1,v2 = γ L−1 × (zv1zt1zt2 . . . zv2). (3)
γ is a discount factor that takes into account the length of the walk in influencing the alignment between v1 and v2. Averaging z̃v1,v2 across several random walks provides an estimate of the effective alignment. z̃ approximates w from Eq. 2. Therefore, we can now minimize:
min h1,...,h|V̂ |
∑
{v1,v2}∈V̂ d(v1, v2)× z̃v1,v2 . (4)
Since the loss function in Eq. 4 is non-convex but smooth, we solve for an optimal solution using
an iterative method: in every iteration, we sample N random walks per node and the node embedding update is computed using the gradient of the empirical loss. See Appendix Sec. A.1 for details of the parameter gridsearch. Actor groups are generated by clustering TAMPA-trained node embeddings via HDBSCAN (McInnes et al., 2017).

heading: 5 Evaluation and Discussion
text: To assess the effectiveness of our framework, we use a two-step evaluation approach. First, we rate the quality of the alignment relationships in the INCANT networks. Second, we evaluate the actor groups generated by TAMPA. It is important to note that the inter-actor relationship quality directly impacts the quality of the resulting actor groups. For further reference, we have attached the codebase and supplemental network files. You can access them in our repository at the following link:3.

heading: 5.1 INCANT alignment relationships correlate to human perception
text: Tab. 4 summarizes the performance of alignment relationship extraction with respect to ground truth labeling performed by MTurk workers (on a random subset of alignment relationships). Details about the labeling setup are provided in Appendix Sec. B. The accuracy, as well as the precision, recall and F1 scores (macro) are high (> 0.75), suggesting good correspondence between the two label sets. Note that in addition to demonstrating that INCANT relationships are accurate, this high performance is indicative of the ability of our QA templates to generalize across domains: The crowd-sourced QA templates in Tab. 1 are not domain-dependent, and yet appear to yield high-fidelity inter-actor alignment relationships for all six evaluated domains.
3Repository: https://osf.io/px3v6
An INCANT subnetwork for the Gun Regulations domain is presented in Fig. 3. An actor’s NER tag corresponds to node color, and the question template responsible for an alignment relationship is displayed along the edge. The color intensity of each edge – blue (aligned) or red (disaligned) – is proportional to the corresponding question template’s score (see Tab. 1)4.
Actor alignments are immediately observed: “donald trump” and “ron desantis” are aligned as both actors support the “second amendment”, and live and campaign in the same state (“florida”). Alignments are transitive: {maga republicans, biden} and {biden, the second amendment} are disalignments suggesting maga republicans, second amendment are aligned. TAMPA automates this discovery process.

heading: 5.2 Evaluating actor groups discovered by TAMPA
text: We first visualize whether the actor groups returned by HDBSCAN are well-separated in the embedding space. As seen in Fig. 4, even the simple measure of pairwise cosine distance is able to separate the clusters. We perform human evaluation (using AMT workers) to evaluate the quality of the actor groups with respect to two baselines. B1 Community detection: We construct communities in the INCANT network Ĝ by using the Louvain algorithm (Blondel et al., 2008). Recall that the nodes correspond to actors and each edge corresponds to the question template connecting a pair
4All INCANT networks are in the OSF database.
of actors. The alignment scores from Tab. 1 are used as the weights along the edges;
B2 Naïve density-based clustering: Densitybased clustering methods such as HDBSCAN can identify clusters provided an adjacency matrix that contains the pairwise distance metric (> 0) between every pair of points. We use the alignment scores for existing edges (similar to B1) and replace absent distance values (missing edges in the INCANT network) with a small positive value (2). We replace negative values (between disaligned actors) with a large positive value (10).
In a blind survey, MTurk workers choose the best of three partitionings – B1, B2 and TAMPA – of actors into groups. Labeling setup details are discussed in Appendix Sec. B. Results are presented in Tab. 5: MTurk workers chose TAMPA actor groups to others (baseline ACC = 0.33). Why do we need TAMPA?
Recall that TAMPA was designed to generalize inter-actor alignments beyond the sparse set of direct relationships available in the INCANT networks (sparsity of relationships indicated in Tab. 3). To illustrate TAMPA’s operation, we consider the following example involving an actor group identified from the Recession Fears domain (narrative description in italics):
{federal reserve, jerome powell, new york, janet yellen}→ treasury moves to curb inflation;
In the corresponding INCANT network Ĝ, there is no direct link between two familiar actors “Janet Yellen” and “Jerome Powell”. This is not surprising since our question templates only involve single entities and do not account for multi-entity question, such as “Where have both Janet Yellen and Jerome Powell worked together?”. The message-passing scheme in TAMPA addresses precisely this limitation by approximating alignment relationships transitively. Consider two alignments that are present in the INCANT network: » “Janet Yellen” (PER) → Who does {phrase} support? → “the Federal Reserve” ∧ » “Jerome Powell” (PER) → Where did {phrase} work? → “the Federal Reserve”
Observe that in these constraints, single-entity QA conveys alignment information with a shared tertiary entity: Yellen *supports* the Federal Reserve and Jerome Powell *works* at the Federal Reserve. TAMPA’s message-passing algorithm is incentivized to iteratively refine Yellen and Powell’s node representations to be close to that of the
Federal Reserve, and effectively construct a strong alignment relationship between the pair:
=⇒ “Janet Yellen”, “Jerome Powell” are aligned.

heading: 5.3 Ablation Study: Performance of TAMPA as a function of INCANT network structure
text: TAMPA relies on the network structure of Ĝ to model the effective alignment between every pair of nodes. We evaluate the extent of this dependence by constructing a modified network baseline, PERM: Ĝ edges are shuffled while maintaining a constant average node degree. Performance of TAMPA on the INCANT network is compared to the PERM baseline by: (a) Evaluating the separation of actor group clusters using unsupervised metrics; and (b) Visualizing the generated actor groups. As for (b), the random edge shuffling predictably worsens the quality of actor groups since the ground truth alignment information is intentionally corrupted (see Fig. 6 in the Appendix for examples).
For (a), we compute the Silhouette score histogram (Rousseeuw, 1987) ∈ [−1, 1] after clustering TAMPA-trained node embeddings for both INCANT and PERM: a node’s score correlates to its membership strength within its actor group. Strength is computed using the pairwise cosine
distance between the trained embeddings. In Tab. 6, the distribution statistics for the INCANT vs. PERM networks are compared: pKS, the p-value of the Kolmogorov-Smirnov (KS) test (Hodges, 1958) compares the shape of the INCANT vs. PERM histogram distributions. p < 0.05 implies the null hypothesis is false, i.e the two Silhouette score distributions are not identical. Within each distribution, we compute the mean µ, confidence interval (CI) and the interquartile range (IQ) (Whaley III, 2005): INCANT networks consistently produce a larger µ (close to 1) and smaller IQ, evidence of a score distribution that skews toward 1, and indicative of better separated clusters.

heading: 6 Concluding Remarks
text: In this work, we propose a novel approach for identifying aligned actors and actor groups from the mixture of latent narratives that undergird domainconditioned free-form text. The success of our approach is evaluated using both qualitative (see Figs. 3,4 and 5) and quantitative (see Tabs. 4,5 and 6) evidence. We show in Fig. 5 that these groups can be used to assemble corresponding narrative networks that convey “my side, your side and the evidence” supporting each side.
When these narrative networks are viewed jointly, we observe a struggle for narrative dominance. In many cases, the tactics proposed in one narrative to counter external threats become threats in and of themselves in other narratives. For instance, in Fig. 5, the relationship tuple “biden administration” → looking to pass → “a ban on assault weapons” (top right) is a strategy to counter gun violence, a threat. Conversely, this same strategy is perceived as a threat by gun rights activists.
This example highlights the complexity of the narrative landscape and how the same inter-actor relationship can take on distinct, often conflicting roles, depending on the side we choose.
Limitations
Key limitations are listed: (a) Inter-actor networks (from Sec. 4.1) are structured representations of the input data. Since the dataset is assembled ondemand from GDELT, the recall of information given a particular domain depends on its popularity at that time. (b) The TAMPA message passing algorithm (from Sec. 4.2.1) is iterative and converges to a local optimum that may perform poorly with human evaluation for particular domains. (c) The various Transformer models – COREF (from Sec. 3), NER, QA (from Sec. 2.1) – can occasionally produce false positive results. The autoregressive coreference resolution in particular occasionally fails to resolve long-range dependencies across segments, which in turn decreases the recall of nodes and edges from the data. (d) The end-to-end model is only validated for the en_US locale since the Transformer models utilized in the work are most performant in English and many conversation domains are country-specific. (e) In the TAMPA algorithm, actors with a higher degree in Ĝ are associated to a higher quality of node embeddings since there are more inter-actor alignment constraints. (f) TAMPA uses HDBSCAN as a clustering algorithm: as with any unsupervised ML algorithm, some clusters are more diffuse than others.
Ethics Statement
Process: The datasets used in this analysis were obtained from GDELT, an open-access platform that indexes world news. The scraped dataset is provided in a processed network format, after bestin-class removal of Personally Identifiable Information (PII). Data and codebases are accessible in the OSF repository (https://osf.io/px3v6/). Future Use: The resulting alignment networks generated by our framework are a representation of the datasets identified on-demand from GDELT. If the sources from GDELT are/or become highly biased to specific news sources, the resulting networks would become biased as well. In this case, the addition of more data sources might be necessary. Additionally, use of this tool in an unmoderated fashion may inhibit free-speech, profile social media users and empower surveillance efforts.

heading: A TAMPA
text: A.1 Training details
Node embeddings are randomly initialized (h ∈ RD, D = 3). The length of each random walk N is 10 and γ = 0.95. Batch size b = 10 (the number of random walks considered per node per iteration), number of iterations M = 20K. We apply simulated annealing during the learning process: nodes are randomly perturbed with probability h = 1− iM . Parameter set is presented in Table 7.
A.2 TAMPA on PERM baseline
See Tab. 7 for the hyperparameters considered for the message passing algorithm TAMPA. The best parameter set is in bold. Models were trained on a 64-core server with 2 TITAN RTX GPUs running Question-Answering and Co-reference Resolution in tandem. Training time per domain does not exceed 1.5 hours.
B Instructions to Amazon Mechanical Turk workers
For both Amazon Mechanical Turk (AMT) tasks described below, workers were required to be Masters-granted (https://www.mturk.com/help), present in the en_US locale. Surveys were hosted on-prem and LabelStudio (Tkachenko et al., 2020-2021) was used for creating the survey templates. The post-processed datasets are available for download from the OSF repository (https://osf.io/px3v6/?view_only= b9223fba3e3d4fbcb7ba91da70565604) and are meant for research use with CC BY 4.0 licensing. Workers were paid $5 for 45 minutes of annotation time. Our estimated time-to-completion was 25 minutes. An overview of the 2 labeling tasks were presented up front to annotators on the AMT platform (see Fig. 7).
B.1 AMT Task 1: Evaluating the quality of alignment relationships
In Fig. 8, we show a snapshot of the instructions presented to MTurk workers to classify a pair of actors present within a context window of text as aligned or disaligned. Each worker was allowed to label at most 50 samples of the dataset and was allotted 2 hours for the survey. Annotated samples from each worker were randomly sampled and manually verified to ensure quality.
B.2 AMT Task 2: Evaluating the quality of actor groups
MTurk workers are given a preliminary survey to guarantee that they possess sufficient domain knowledge in order to accurately identify the actors that form the actor groups, and to evaluate
whether the actors belonging to each group believe in similar worldviews given the conversation domain. To increase an MTurk worker’s chances of being able to identify the actors, we pre-select the top-K = 25 actors (by degree) from Ĝ and their corresponding actor groups. Clustering was performed using the N = 100 highest-degree nodes from Ĝ. Fig. 9 shows the instructions provided to the MTurk workers. Once again, annotated samples from each worker were randomly sampled to ensure quality. In total, annotators labeled 240 samples – 40 per domain.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: After Concluding Remarks; before References Pg 9. Section* number unmarked.

heading: 3 A2. Did you discuss any potential risks of your work?
text: After Concluding Remarks; before References Pg 9. Section* number unmarked.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract - Pg 1, Background and Motivation (Introduction) - Pg 1-2
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Sec. 3 - Data Collection (Data), Sec. 4 - Methods (Code)

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Sec. 3 - Data Collection (Data), Sec. 4 - Methods (Code): Python libraries were used when applicable for data processing and model training. These are referenced in text.

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Sec. 3 & See Hyperlink 3.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Appendix Sec. B (Pg. 11)

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Sec. 3

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Sec. 3, Appendix Sec. B
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Sec. 3, 4. Tab. 3
C 3 Did you run computational experiments? Sec. 4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Sec. 4, A.2
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Sec. 4, A.2

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Sec. 6 (6.1, 6.2, 6.3)

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Sec. 3, 4, 5
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Sec. 6.1, 6.2

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Appendix Sec. B

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Sec. 6.1, Appendix Sec. B

heading: 3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to
text: crowdworkers explain how the data would be used? Sec. B.1, B.2
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: Appendix Sec. B, Sec. 3 (for context)

heading: Future Work
text: Future Use: The resulting alignment networks generated by our framework are a representation of the datasets identified on-demand from GDELT. If the sources from GDELT are/or become highly biased to specific news sources, the resulting networks would become biased as well. In this case, the addition of more data sources might be necessary. Additionally, use of this tool in an unmoderated fashion may inhibit free-speech, profile social media users and empower surveillance efforts