Clustering Document Parts: Detecting and Characterizing Influence Campaigns from Documents (2024)

Abstract: We propose a novel clustering pipeline to detect and characterize influence campaigns from documents. This approach clusters parts of document, detects clusters that likely reflect an influence campaign, and then identifies documents linked to an influence campaign via their association with the high-influence clusters. Our approach outperforms both the direct document-level classification and the direct document-level clustering approach in predicting if a document is part of an influence campaign. We propose various novel techniques to enhance our pipeline, including using an existing event factuality prediction system to obtain document parts, and aggregating multiple clustering experiments to improve the performance of both cluster and document classification. Classifying documents after clustering not only accurately extracts the parts of the documents that are relevant to influence campaigns, but also captures influence campaigns as a coordinated and holistic phenomenon. Our approach makes possible more fine-grained and interpretable characterizations of influence campaigns from documents.

## introduction inspired by @xcite and @xcite , we define an influence campaign as a coordinated and strategic effort to shape and manipulate the perceptions of a target audience about certain things or issues over a period of time. it can be organized by an individual, organization, or government for various purposes, such as promoting a specific public image, product, policy, or political narrative. it can be carried out through various channels, including traditional media and online platforms. consequently, detecting an influence campaign requires holistic evaluations and the use of multiple indicators, such as the social network @xcite , that point to a collective effort with a shared motive that aims to impact public opinions in a certain way. accurate and reliable detection typically involves extensive manual verification by domain experts, taking into account both textual and non-textual information @xcite . in the context of nlp, detecting influence campaigns typically means predicting if an input document is part of an influence campaign @xcite , i.e., a binary classification task. however, this is a different task from capturing the phenomenon of influence campaigns, which naturally is a clustering problem, i.e., grouping a collection of documents that reflect an influence campaign. in practice, the classification task is difficult if not doomed, because by definition, an influence campaign cannot possibly be inferred from a single document. consider the examples in media positive negative twitter ...putin cleans up the bioweapons labs installed by the deep state... (44 tks) ...rt @emmanuelmacron france strongly condemns russia's decision to wage war on ukraine... (19 tks) forum ...a secret nato laboratory for biological weapons...biological weapons tests were carried out in the laboratories of this facility... (638 tks) ...[nato] has blocked ukraine's plan to enter...item 3: ukraine was a pawn that the westerners deliberately sacrificed to strengthen nato... (703 tks) news ...a nato secret biological laboratory with biological weapons...the biological laboratory under the azovstal plant in marioupol in the so-called pit-404 facility was built...in the laboratories of the facility, tests were carried out to create biological weapons... (1497 tks) ...russia's demand for neutrality...but nato members said that ukraine's membership was at best a distant option... [the leader of the ukrainian separatist region of lugansk said he could hold a referendum on integration into russia,] a decision immediately criticized by kiev...(1152 tks) in this paper, we propose a novel text-only clustering-based pipeline to help detect and characterize influence campaigns from documents. unlike the typical document-level classification approach discussed above, the pipeline predicts influence campaigns directly on the cluster level, i.e., it predicts whether a cluster of document parts present an influence campaign (a high-influence cluster). from there, the pipeline further predicts whether any document associated with a highinfluence cluster is part of the influence campaign via a dynamic projection procedure. as a result, our pipeline is capable of handling the two aspects of the influence campaign detection task: capturing influence campaigns as a holistic phenomenon, and predicting documents that are part of an influence campaign. since influence campaigns are captured by clusters of document parts and the documents predicted to engage in an influence campaign are projected from these clusters, our pipeline enables fine-grained and interpretable characterizations of influence campaigns from documents. the specific contributions of this paper are as follows. this paper is structured as follows. we review related works in sec 2 and motivate and explain the algorithmic idea underlying our novel clustering pipeline in sec 3. we describe the influence campaign detection dataset we test our pipeline on in sec 4 and the experiments in sec 5. the results are discussed in sec 6. the paper concludes in sec 7. we release our code at https://github.com/ jaaack-wang/detect-influence-campaigns . ## related work there have been very few studies in the existing literature that approach influence campaigns in the general sense as we define it in sec 1. the influence campaigns studied in most previous research (des @xcite @xcite are political influence campaigns, or some closely related political influence operations that may be an influence campaign, such as the spreading of mis/dis-information @xcite @xcite @xcite @xcite . the most common detection method relevant for influence campaigns is the detection of bots in social networks @xcite @xcite @xcite . for text-based influence campaign detection, various nlp methods have been explored. for example, a recent study leverages llms @xcite to predict if a tweet is part of an known influence campaign. other studies relevant for influence campaigns utilize various sources of linguistic features (e.g., lexicon counts, ngrams, word embedding) to train or fine-tune different models (e.g., bert, graph neural network, decision tree) with a goal to detect propagandistic, deceptive, or misleading information @xcite @xcite . to the best of our knowledge, we are not aware of any study that aims to detect influence campaigns on the cluster level. several twitter datasets have been used by recent studies on detecting influence campaigns (des @xcite , such as the 2016 us election dataset @xcite , data from twitter's information operations archive foot_1 , and russian troll accounts for 2016 us election released by the u.s. congress @xcite . there are also other relevant datasets in other media forms, such as factoid @xcite collected from reddit and proppy @xcite collected from news articles. we note that for all of these datasets that come with labels, the labels are typically created on the basis of some simplistic association or assumption. for example, tweets are assumed to be linked to an influence campaign if they come from russian troll accounts @xcite . to the best of our knowledge, we do not know of any publicly available influence campaign datasets that contain more than one media type. ## pipeline: the algorithmic idea given the coordinated nature of influence campaigns, an influence campaign can be thought of as a cluster of documents that spread a certain theme aimed to influence the target audience. our pipeline follows exactly this intuition and transforms the task of influence campaign detection into one that detects clusters that are highly likely to reflect an influence campaign (i.e., high-influence clusters). then the next step naturally becomes how to accurately select documents (i.e., high-influence documents) associated with the high-influence clusters that reflect an influence campaign, assuming the clusters may contain some noise or false positives. more concretely, our pipeline consists of the following four steps. ## data we use data collected during a large research program, darpa incas project foot_2 . we expect the data to be made public after the end of the research program. we use this dataset as we are not aware of any other datasets that have expert-verified annotations indicating if a collection of documents contain influence campaigns. the data contains four piles of online posts published during january 31 to june 30, 2022. each pile is a collection of documents in six media forms, namely, twitter, forum, news, blog, reddit, and other. two of the four piles contain documents that engage in an influence campaign that spreads disinformation related to ukraine bioweapons conspiracy theory (see the majority of these documents are written in french, typically accompanied by an english translation, and the rest are in english. we choose to work on the translated french portion of the data. this portion has over 8 times more documents than the english subcorpus, but with a significantly smaller portion of documents linked to an influence campaign, less than 8%. we believe this represents a more realistic and challenging setting for detecting influence campaigns from documents. given the overall small size of the dataset, we randomly split the french data into two parts, the train and test sets, but the train set can be further split for training and validation where needed. we split at the document level, with a ratio of 80/20, as shown in ## conclusion we have presented a new approach to finding influence campaigns, which relies on four core features: there are several avenues for possible future work and we list three below. (1) datasets. given the increasing importance of detecting influence campaigns, we hope there will be more datasets annotated on the document collection level for an influence campaign. (2) incorporating non-textual information. our current pipeline is a text-only system. leveraging non-textual information, such as social interactions and the authors' past activities, may help us create a more complicated and comprehensive system (e.g., using graph neural network) that enhances the accurate and reliable detection of influence campaigns. however, such work cannot be possible without good datasets. (3) automatic characterization of influence campaigns. our work captures influence campaigns by the highinfluence clusters, which may contain a large number of semantically related document parts, possibly with noise. to fully make sense of these clusters, we need to have some automatic ways of characterizing them in a fine-grained and interpretable way aligned with the downstream needs. our preliminary experiments show that llms may be a potential option. university.