N ews S ense: Reference-free Verification via Cross-document Comparison (2023)

Abstract: We present NewsSense, a novel sensemaking tool and reading interface designed to collect and integrate information from multiple news articles on a central topic. NewsSense provides “reference-free verification,” augmenting a central grounding article of the user’s choice by: (1) linking to related articles from different sources; and (2) providing inline highlights on how specific claims are either supported or contradicted by information from other articles. Using NewsSense, users can seamlessly digest and cross-check multiple information sources without disturbing their natural reading flow. Our pilot study shows that NewsSense has the potential to help users identify key information, verify the credibility of news articles, explore different perspectives, and understand what content is supported, contradicted, or missing.

## introduction why is it so hard, and so exhausting, to read the news? in the quest for knowledge, news readers today must contend with a rapidly evolving 24-hour news cycle, multiple news venues competing for attention and clicks, and the challenge of integrating fact-based reporting, opinion pieces, and social media commentary @xcite @xcite . with news becoming increasingly politicized @xcite readers also face the challenge of identifying and avoiding misinformation, disinformation, and hyperbolic "clickbait" as they try to remain informed about the world around them. various solutions have been proposed to assist with users' news reading. for example, media watchdog companies have created media bias charts to represent political leaning and credibility of news sources 1 2 . however, these resources force users to rely on potentially untrustworthy third-party designations of media bias, which treat each news source as a whole, without digging into specific articles or topics. while novel automatic fact checking @xcite and fake news detection @xcite systems can provide verification per-article, these approaches typically rely on a preordained corpus of verified facts which cannot keep up with the always-evolving facts, and may not reflect user preferences or multilateral perspectives. aggregating articles from heterogeneous sources seem a more promising direction for cross-checking new facts (without predetermined groundtruths) and collecting different perspectives, but existing attempts are still too coarse and over-1 https://www.allsides.com/media-bias/media-bias-ratingmethods 2 https://adfontesmedia.com/interactive-media-bias-chart/ whelming. for example, both google news' "stories" foot_0 feature and ground.news foot_1 collect articles about the same events but display them in the form of exhaustive lists -users are still forced to read and compare each article on its own. we argue that instead of simply collecting and aggregating news articles, information and claims from multiple sources should be integrated in a way that allows users to identify fine-grained claimlevel bias, spin, controversy, or evidence. we present newssense, a novel framework for sensemaking within a cluster of documents, to address the three key problems of news reading -bias, factuality, and article overload -in a single streamlined interface. newssense leverages existing modular natural language processing (nlp) techniques to identify and link claims made across a cluster of news articles, such that these articles become references for each other. newssense also displays the linking information using an interactive reading interface, which allows users to easily explore the cross-document connections without being overwhelmed. in pilot user studies, we see that the newssense framework has the potential to help users identify key information, verify the credibility of news articles, and explore different perspectives. while newssense is primarily implemented for news articles, our framework can be easily generalized to other assisted reading and cross-checking scenarios (e.g., compare multiple manuscripts in literature reviews). the key contributions of newssense are: 1. a pipeline for analyzing the connections between a collection of documents. 2. a two-stage method for efficiently computing cross-document links between claims that support or contradict each other, enabling "reference-free" fact checking. 3. a framework for visualizing cross-document connections, and integrating claims from multiple documents into a single reading experience. we conclude by discussing the generality and potential social benefits of newssense. ## related work this section covers related research across media analytics, sensemaking, and natural language processing. though some core ideas of this work have been explored in the past, to our knowledge they have never been combined in a single system. media bias and analytics research on media bias includes academic research to study social media sharing patterns @xcite and bias within media publications @xcite @xcite . commercial products exist in this area as well, such as the media bias charts of allsides foot_2 , which classifies political slant into one of five categories, and ad fontes media foot_3 , which models both political slant and factual credibility. research on news and social content aggregation has focused primarily on headline detection, timeline construction and clustering @xcite , and event detection @xcite . there exist user-oriented products in this space, such as google news stories foot_4 , and ground.news foot_5 . some outlets, such as propublica, aggregate their news stories into timelines foot_6 . ## the newssense framework the core philosophy behind newssense is to go beyond article aggregation by integrating the information contained within a cluster of related news articles into the reading experience. newssense starts with a single "focus" article and a set of related "background" articles. the distinction between focus and background article is arbitrary, as any article within the cluster could be designated the focus article. newssense then identifies claims within the focus article that are related -either by contradiction or entailment -to claims within the background articles. the claims in the focus article are then highlighted, and linked to the background articles so that users can explore the supporting or contradicting evidence for a given claim without just relying on third-party measurements of bias or credibility. the newssense interface has three primary components: a focus article, sentence highlights, and external evidence. together, these elements display the computed connections between the focus article and the background articles. ## pilot study to gather feedback on the proposed newssense interface and provide insights for the actual implementation, we conducted a pilot user study using a newsreader mockup built with figma foot_7 . this section describes the design and results of the study. ## system overview following our user study, we implemented the newssense framework as a browser plugin, which adds augmentations to news articles encountered on the web. figure 1 shows the final appearance of the browser plugin. open source code for the system and plugin, as well as a demo video, can be accessed at github.com/jmilbauer/newssense. this section provides a description of the natural language processing system which powers newssense. figure 3 illustrates the four general steps of the pipeline: collection, selection, filtering, and linking. ## discussion newssense provides an intuitive and effective interface for integrating information from a large cluster of news articles into a single, focused reading experience. although applied in this demo to news articles, the newssense framework could just as easily be applied to the analysis of other types of document clusters as well. the pipeline itself is highly modular, and can easily adopt advancements in nlp technologies to increase the accuracy or decrease processing time. ## conclusion we presented a novel framework for sensemaking within a cluster of documents. we applied this framework to news articles, building newssense, an interactive tool that links claims within one document to supporting or contradicting evidence across the entire document cluster. newssense assists readers by helping them to understand the connections and perspectives across many documents. readers can thus attain a more comprehensive understanding of a given subject, while avoiding the dangers of information overload. crucially, newssense provides a framework for referencefree fact verification, which is essential in domains such as the news where events evolve in real time, because a knowledge source for factual grounding may not be available. our work expands the growing body of literature on natural language processing applications to document-level sensemaking by demonstrating the utility of automatically generated cross-document links, as well as the application of sensemaking tools to the news reading experience.