Wolfies at S em E val-2022 Task 8: Feature extraction pipeline with transformers for Multi-lingual news article similarity (2022)

Abstract: This work is about finding the similarity between a pair of news articles. There are seven different objective similarity metrics provided in the dataset for each pair and the news articles are in multiple different languages. On top of the pre-trained embedding model, we calculated cosine similarity for baseline results and feed-forward neural network was then trained on top of it to improve the results. We also built separate pipelines for each similarity metric for feature extraction. We could see significant improvement from baseline results using feature extraction and feed-forward neural network.

## introduction for finding similarity between two documents @xcite , the general approach is to get the embeddings for the documents and use some similarity metric like cosine similarity to get similarity measures. such work has been done in multiple domains like research papers, semantic similarity @xcite document similarity @xcite and for news articles @xcite ) @xcite . there have been other works to find similarity between documents of multiple languages @xcite as well. the most significant difference here is that authors have used subjective similarity measures like writing style and authorship for these works. we do feature extraction corresponding to each similarity metric via a unique feature extraction pipeline. for example, for geolocation, we extract the locations from the text. to make it a domainspecific task, we train a network on top of the generic similarity pre-trained embedding model. to test our approach, we use a pre-trained model for sentence similarity based on s-bert @xcite . we calculate cosine similarity @xcite as our baseline on the embedding of the documents. we train a feed-forward network on top of the pre-trained embedding to learn about our domain-specific task. we also use a large pretrained model, which can take more words as input and compare the results. our approaches result in an almost 60% mse improvement over the baseline model. our contributions are mentioned below: ## conclusions we created an end-to-end pipeline to find similarities between two news articles in this work and used multiple approaches to find the most optimal way to calculate similarity. we were able to deal with the problem of multilingualism best using a multilingual s-bert model. we were able to identify the shortcomings of the pre-trained model for similarity in the location metric. we also explored using a large model, which did not significantly improve the results. on the eval data, the multilingual model showed decent results even on unseen languages, which shows that we can extend the model to other languages with minor changes.