NSTM : Real-Time Query-Driven News Overview Composition at B loomberg (2020)

Abstract: Millions of news articles from hundreds of thousands of sources around the globe appear in news aggregators every day. Consuming such a volume of news presents an almost insurmountable challenge. For example, a reader searching on Bloomberg’s system for news about the U.K. would find 10,000 articles on a typical day. Apple Inc., the world’s most journalistically covered company, garners around 1,800 news articles a day. We realized that a new kind of summarization engine was needed, one that would condense large volumes of news into short, easy to absorb points. The system would filter out noise and duplicates to identify and summarize key news about companies, countries or markets. When given a user query, Bloomberg’s solution, Key News Themes (or NSTM), leverages state-of-the-art semantic clustering techniques and novel summarization methods to produce comprehensive, yet concise, digests to dramatically simplify the news consumption process. NSTM is available to hundreds of thousands of readers around the world and serves thousands of requests daily with sub-second latency. At ACL 2020, we will present a demo of NSTM.

## introduction in many domains, finding contextually-important news as fast as possible is a key goal. with millions of articles published around the globe each day, quickly finding relevant and actionable news can mean the difference between success and failure. when provided with a search query, a traditional system returns links to articles sorted by relevance. however, users typically encounter (near) duplicate or overlapping articles, making it hard to quickly identify key events and easy to miss less-reported stories. moreover, news headlines are frequently sensational, opaque, or verbose, forcing readers to open and read individual articles. for illustration, imagine an analyst sees the price of amazon.com stock drop and wants to know why. with a traditional system, they would search for news on the company and wade through many stories (307 in this case foot_0 ), often with duplicate information or unhelpful headlines, to slowly build up a full picture of what the key events were. by contrast, using nstm (key news themes), this same analyst can search for 'amazon.com', over a given time horizon, and promptly receive a concise and comprehensive overview of the news, as shown in fig. 1 . we tackle the challenges involved with consuming vast quantities of news by leveraging modern techniques to semantically cluster stories, as well as innovative summarization methods to extract succinct, informational summaries for each cluster. a handful of key stories are then selected from each cluster. we define a (story cluster, summary, key stories) triple as one theme and an ordered list of themes as an overview. nstm works at web scale but responds to arbitrary user queries with sub-second latency. it is deployed to hundreds of thousands of users around the globe and serves thousands of requests per day. ## design goals we focus on the scenario where a news search query can render many matching news articles, from tens up to hundreds of thousands. the task is to create a succinct overview of the results to help our users to easily grasp the gist of them without combing through the individual articles. since the matching articles often cover various aspects and events, nstm must first cluster related stories to form a clear separation among them. furthermore, the system must extract a concise (up to 50 characters, or roughly 6 tokens) summary for each cluster. it needs to be short enough to be understandable to humans with a single glance, but also rich enough to retain critical details from a minimal 'who-does-what' stub, so the most popular noun phrase or entity alone will not suffice. such conciseness also helps when screen space is limited (for context-driven applications or mobile devices). from each cluster, nstm must surface a few key stories to provide a sample of its contents. the clusters themselves should also be ranked to highlight the most important few in limited screen space. finally, the system must be fast. it may only take up to a few seconds for the slowest queries. ## related work a comparable system is google news' 'full coverage' feature foot_1 , which groups stories from different sources, akin to our clustering approach. however, it doesn't offer summarization and its clustered view is unavailable for arbitrary search queries. summa @xcite is another comparable system which integrates a variety of nlp components and provides support for numerous media and languages, to simultaneously monitor several media broadcasts. summa applies the online clustering algorithm by @xcite and the extractive summarization algorithm by @xcite . in contrast to nstm, summa focuses on scenarios with continuous multimedia and multilingual data streams and produces much longer summaries. ## demonstration nstm was deployed to our clients in 2019. using the ui depicted in fig. 1 , users can find overviews for customized queries to help support their work. from this screen, the user can enter a search query using any combination of boolean logic with tagor keyword-based terms. they may also alter the tables 1 and 2 show example theme summaries generated for the queries 'facebook' and 'u.k.'. note that the summaries are quite different from what has previously been studied by the nlp community (in terms of brevity and grammatical style) and that they accurately represent distinct events. in addition to user-driven settings, nstm can be used to supplement context-driven applications. one example, demonstrated in appendix d, uses themes provided by nstm to help explain why companies or topics are 'trending'. ## conclusion we presented nstm, a novel and production-ready system that composes concise and human-readable news overviews given arbitrary user search queries. nstm is the first of its kind; it is query-driven, it offers unique news overviews which leverage clustering and succinct summarization, and it has been released to hundreds of thousands of users. we also demonstrated effective adoption of modern nlp techniques and advances in the design and implementation of the system, which we believe will be of interest to the community. there are many open questions which we intend to research, such as whether autoregressivity in neural sentence compression can be exploited and how to compose themes over longer time periods.