Interesting cross-border news discovery using cross-lingual article linking and document similarity (2021)

Abstract: Team Name: team-8 Embeddia Tool: Cross-Lingual Document Retrieval Zosa et al. Dataset: Estonian and Latvian news datasets abstract: Contemporary news media face increasing amounts of available data that can be of use when prioritizing, selecting and discovering new news. In this work we propose a methodology for retrieving interesting articles in a cross-border news discovery setting. More specifically, we explore how a set of seed documents in Estonian can be projected in Latvian document space and serve as a basis for discovery of novel interesting pieces of Latvian news that would interest Estonian readers. The proposed methodology was evaluated by Estonian journalist who confirmed that in the best setting, from top 10 retrieved Latvian documents, half of them represent news that are potentially interesting to be taken by the Estonian media house and presented to Estonian readers.

## introduction this paper presents our results of the participation in the hackaton, which was organised as part of the eacl 2021 hackashop on news media content analysis and automated report generation. we are addressing the embeddia hackathon challenge on identifying interesting news from neighbouring countries @xcite in estonian and latvian context, which is a fully novel document retrieval task performed on recently released em-beddia news datasets. estonian journalists are very interested in identifying stories from latvia, which will attract a large number of readers and are "special". while performing keyword-based search for latvian news, where estonians are mentioned is a simple task, this challenge on the contrary aims to identify a small set of documents from a larger number of topics, e.g. scandals, deaths and gossip that might be somehow connected to estonia: not only by mentioning estonians but by identifying news and stories that estonians relate to (for example, when similar things have happened in estonia or when similar news have been popular in estonia). in our approach, we first automatically create a collection of interesting articles using a stringbased search and cross-lingual document linking, and then rank the query documents based on the proportion of interesting documents in their neighbourhood (where the neighbourhood is defined by a document similarity) by the newly introduced seed news of interest score @xcite . the article first presents the datasets (section 2), introduces the methodology (section 3), and presents our experimental results (section 4). the code and the data are made publicly available (see section 5). finally, section 6 concludes the paper and presents the ideas for further work. ## datasets in this study, we used the following resources. ## methodology our methodology consists of two steps. first, we automatically construct the datasets of interesting latvian articles and next propose a method to retrieve interesting articles by ranking a given query document based on the the proportion of interesting articles in its neighbourhood. ## availability the code and data of the experiments is made available on the github: https://github.com/bkolo sk1/interesting-cross-border-news-discov ery 6 conclusion and future work in this work we tackled the problem of retrieving interesting news from one country for the context of another neighbouring country. we focused on finding interesting news in latvian news space that would be engaging for the estonian public. we used latvian and estonian embeddia datasets to construct the document space. first we used a string matching approach to identify a subset of news in estonian media that originated from latvian news. next, we utilized the methods for ad hoc cross lingual document retrieval to find corresponding articles in the latvian news space. after automatically retrieving this set of latvian news articles of interest, we used this information in a novel metric defined as snir, that analyses a news article's neighbourhood in order to measure it's relevance (interestingness). the assumption of the metric is that if the surrounding documents of a query point are relevant, this new point might be of relevance. the snir scores of randomly selected 20 documents and 20 documents identified as examples of interesting news by an estonian journalist showed that their value differ, which is for the further work we propose exploring the keywords appearing in the clusters of interesting news and exploiting their named entity tags in order to achieve even better performance. we also want to include background knowledge from knowledge graphs to improve the document similarity evaluation. special attention will also be paid to setting a threshold for snir which would allow for real-time investigation of best candidates in a real journalistic practice.