Identifying Informational Sources in News Articles (2023)

Abstract: News articles are driven by the informational sources journalists use in reporting. Modeling when, how and why sources get used together in stories can help us better understand the information we consume and even help journalists with the task of producing it. In this work, we take steps toward this goal by constructing the largest and widest-ranging annotated dataset, to date, of informational sources used in news writing. We first show that our dataset can be used to train high-performing models for information detection and source attribution. Then, we introduce a novel task, source prediction, to study the compositionality of sources in news articles – i.e. how they are chosen to complement each other. We show good modeling performance on this task, indicating that there is a pattern to the way different sources are used together in news storytelling. This insight opens the door for a focus on sources in narrative science (i.e. planning-based language generation) and computational journalism (i.e. a source-recommendation system to aid journalists writing stories). All data and model code can be found at https://github.com/alex2awesome/source-exploration.

## introduction journalism informs our worldviews; news itself is informed by the sources reporters use. identifying sources of information in a news article is relevant to many tasks in nlp: misinformation detection @xcite , argumentation @xcite and news discourse @xcite . attributing information to sources is challenging: as shown in ## related work quote attribution prior work in quote attribution has also been aimed at identifying which sources contributed information in news articles. early work explored rules-based methods @xcite and statistical classifiers @xcite to attribute sources to quotes. more recent work has extended these ideas by using bootstrapping to discover new patterns, quootstrap @xcite and training bert-based models on perturbations on these patterns, or quotebert @xcite . one upside of quootstrap and quotebert is that they might adapt better to new domains by learning and generalizing from new patterns. however, the method by which patterns are learned, finding quotes that repeat across outlets, might bias this method towards discovering quotes by oft-quoted figures. these quotes, in turn, may be contextualized differently than other quotes, introducing fundamental biases in which sources get discovered. we urge more consideration of these potential biases, not only for performance considerations but fairness. overall, our work differs from previous work in this field because we defined information more broadly. prior work is quote-focused, whereas we include a larger set of information channels ( persona modeling a second area that our work draws inspiration from is the study of narrative characters and how they are used in fiction. work by @xcite and @xcite used custom topic models to model characters by latent "personas" generated from latent document-level distributions.earlier work extended this topic-modeling approach to news sources @xcite . we see potential for future work merging this with our dataset and framework, using methods like discrete variational autoencoders, which have been applied to document-planning @xcite . ## conclusions we have offered a more expansive definition of sourcing in journalism and introduced the largest attribution dataset capturing this notion. we have developed strong models to identify and attribute information in news articles. we have used these attribution models to create a large silver standard dataset that we used to probe whether source inclusion in news writing follows predictable patterns. overall, we intend this work to serve as a starting point for future inquiries into the nature of source inclusion in news articles. we hope to improve various downstream tasks in nlp and, ultimately, take steps towards building a source recommendation engine that can help journalists in the task of reporting.