Reconstruct to Retrieve: Identifying interesting news in a Cross-lingual setting (2023)

Abstract: An important and resource-intensive task in journalism is retrieving relevant foreign news and its adaptation for local readers. Given the vast amount of foreign articles published and the limited number of journalists available to evaluate their interestingness, this task can be particularly challenging, especially when dealing with smaller languages and countries. In this work, we propose a novel method for large-scale retrieval of potentially translation-worthy articles based on an auto-encoder neural network trained on a limited corpus of relevant foreign news. We hypothesize that the representations of interesting news can be reconstructed very well by an auto-encoder, while irrelevant news would have less adequate reconstructions since they are not used for training the network. Specifically, we focus on extracting articles from the Latvian media for Estonian news media houses. It is worth noting that the available corpora for this task are particularly limited, which adds an extra layer of difficulty to our approach. To evaluate the proposed method, we rely on manual evaluation by an Estonian journalist at Ekspress Meedia and automatic evaluation on a gold standard test set.

## introduction media houses often report relevant foreign news and adapt them to the local readership. with the ever-rising number of published articles and the limited number of people retrieving and curating the stories, the task becomes harder for media houses. the media houses often need to allocate scarce resources available, such as translators of specific languages and journalists, to curate and adapt the stories. in this work, we propose an approach that, given a handful of articles in a given language (estonian), automatically suggests a set of potentially interesting news in a chosen foreign language (latvian), employing a deep auto-encoder network to reconstruct and retrieve the relevant foreign articles. the task of identifying foreign interesting news is defined by the estonian media house, interested in retreival of latvian articles. for example, articles covering international politics (e.g. american elections ) are not interesting, as the estonian house would have other sources for these news. also, many local articles are not interesting, as they are irrelevant for estonians. however, very specific articles are of their interest, including the ones, covering estonians in latvia, topics relevant to estonian readership (e.g. discussion on electronic scooters, doping affairs. these topics are however not predefined, but there is a small dataset of retrieved interesting news. the ratio of interesting to non-interesting news is very small, suggesting the task to be considered as imbalanced classification. in many imbalanced classification tasks (such as phishing detection @xcite , software defect prediction @xcite , wind-turbine @xcite anomaly detection), autoencoders models have been utilized due their ability to reconstruct subgroups of examples well. zhang and zhu (2020) this work extends the previous work on interesting cross-border news retrieval by @xcite , where the authors define a custom metric -snir (seed news of interest ratio). first, the method embeds both the set of interesting articles and the set of candidate articles into a multilingual space @xcite . next, the snir score of each candidate is calculated as the fraction of the interesting articles in a neighborhood of m articles. this metric follows the nearest-neighborbased approach, where they check the ratio of interesting versus non-interesting news in the neighborhood for a given article. if the ratio is bigger than a given threshold, then the article is considered as interesting and thus relevant for translation and adaptation. they define an article as interesting if it is highly relevant to the estonian readership at the time of publishing. to our knowledge, this is the only related work for the addressed task. we extend this work by proposing a novel method, as well as by proposing an automated evaluation setting for our task. in the rest of this paper, section 2 analyzes related work, section 3 describes the data used, followed by the explanation of the proposed method in section 4, and its evaluation in section 5. conclusions and further work are presented in section 6. ## related work in the field of journalism, one of the crucial responsibilities is to search for and gather captivating news stories from neighboring countries. recent research by @xcite examines the use of ontologies, a type of language technology, in the domain of news retrieval. according to their findings, ontologies are primarily used for semantic search in news retrieval systems. additionally, the collaboration between translation and journalism is essential in the process of news retrieval @xcite . machine translation plays a significant role in automatically converting news stories in different languages to a language that is familiar to the news media curator @xcite @xcite @xcite . large language models (llms) are currently at the forefront of the field of machine translation. there are mainly two types of llms: autoregressive and autoencoding. autoregressive models generate text by predicting the next word in a sequence given all the previous words. examples of autoregressive models include gpt-3 @xcite ) -model based on the casual language modeling (clm) task and bert @xcite -model trained with the masked language modeling (mlm) objective. autoencoding models, on the other hand, are trained to reconstruct the original input given a corrupted version of it. these models learn to represent the input in a compact form that captures the most important information. examples of autoencoding models include t5 @xcite and bart @xcite . both types of llms are highly effective for a wide range of tasks in interest to journalism: genre identification @xcite ), text classification @xcite , sentiment analysis @xcite @xcite @xcite , machine translation @xcite @xcite , keyword extraction @xcite and more. moreover, multilingual variants of these models (such as xlmr @xcite ) have been developed to support multiple languages, making them even more useful for cross-lingual nlp tasks. autoencoder networks have found widespread use for input retrieval via reconstruction in various domains. for instance, @xcite developed a siamese autoencoder for dense text retrieval, @xcite benchmarked a network consisting of an autoencoder and a generative adversarial network for zero-shot cross-modal retrieval, reporting promising results. additionally, @xcite investigated the effect of contrastive pre-training for dense retrieval via autoencoder networks and achieved highly favorable outcomes. in this paper, we apply autoencoders to discover interesting news by reconstructing documents. we define interesting news (based on prior work @xcite ) as news that readers relate to and originates from foreign countries. ## data the data used in this work consists of estonian and latvian articles (published in the period between 01.01.2018 until 01.12.2019) by media houses belonging to the ekspress meedia group. we used the following corpora from the embeddia news archives data set @xcite . ## evaluation we evaluate the method in two scenarios, manual and automated. in both systems, we use the testing data for retrieving the top-ranked articles as interesting and relevant. ## conclusion and further work in this work, we have developed an auto-encoderbased approach for detecting and retrieving cross-border news. the method is trained unsupervised, given news datasets in two languages, relevant and non-relevant articles, and potential media houses hot words. the approach is shown to retrieve articles with 40% relevance, as evaluated manually by a media expert, and outperforms random-based approaches through recall@k evaluation. for further work, we suggest exploring the significance of certain topics and keywords in a given time window, hypothesizing that story/topic relevance is time-dependent. we also propose exploring a term-matching approach that considers named entities and keyword matching to rank the relevance of an article. lastly, we suggest investigating how combining the snir and auto-encoder as a weighted rank score could improve retrieval quality. to improve the relevance of retrieved articles, future work could explore the use of user feedback and relevance feedback mechanisms (such as rlhf). by incorporating user preferences and feedback, the system may be able to better tailor its results to the needs and interests of individual users.