Semantic Textual Similarity Impact on NLP Applications

Human has an intrinsic ability to recognize the degree of similarity and difference between texts. Simulating the process of human judgment in computers is still an extremely difficult task. Semantic Textual Similarity (STS) is the task of assessing the degree to which two short texts are similar to each other in terms of meaning. Many natural language processing (NLP) applications rely on assessing the semantic similarity of text segments as a core component to achieve their goals; such as information retrieval, machine translation evaluation, automatic short answer grading, paraphrase identification, recognizing textual entailment, and others. An infinite number of meaningful sentences can be generated in any natural language. Hence, short texts present many challenges in NLP, unlike words and documents. Despite the shortness of a sentence, it can accommodate the most complex forms of human expression. Some pairs of sentences may represent the same meaning, even though there are few matching words between them, while other pairs may have totally different meanings; however, a high word overlap occurs between them. Several approaches have been proposed in the literature to determine the semantic similarity between short texts. The majority of the STS approaches presented recently were supervised approaches, where a machine learning or deep learning technique used with feature engineering. Unsupervised STS approaches are presented as well as a single similarity measure, which are characterized by the fact that they do not require learning data, but they still suffer from some limitations. This thesis aims to provide an effective solution to the problem of semantic similarity between English texts, specifically short texts (sentences), compared to the unsupervised STS approaches presented in the literature. This research objective has been achieved through a list of contributions: (1) Presenting a new synset-oriented word aligner that relies on a multilingual semantic network named BabelNet, which connects a wide range of concepts and named entities with large numbers of semantic relations. (2) Proposing three unsupervised STS approaches, namely String kernel-based (SK), Alignment-based (AL), and Weighted alignment-based (WAL). (3) Proposing an effective unsupervised ensemble STS (ESTS) approach, which incorporates the merits of four similarity measures: our proposed weighted alignment-based (WAL), surface-based, corpus-based, and enhanced edit distance. (4) Proposing a supervised approach that includes the proposed unsupervised measures as separate features. A thorough intrinsic evaluation has been conducted on most of the available standard data sets, namely the SemEval series data sets, and the STS Benchmark data set. The experimental results proved that the proposed unsupervised ensemble approach (ESTS) outperforms the state-of-the-art unsupervised systems over all the evaluation data sets with average Pearson correlation results ranging from 0.6574 (in SemEval2013 datasets) to 0.8459 (for the SemEval2017 datasets), and also the best-supervised systems in some data sets, which is a promising result. Also, the proposed approaches were tested on cross-lingual and multilingual data sets to evaluate their generalizability and their ability to assess the semantic similarity between texts in different languages. The proposed ESTS outperforms the state-of-the-art approach in the Spanish sentence pair, and achieved a Pearson correlation results ranging from 0.55 (in Turkish-English cross-lingual sentence pair) to 0.85 (for the Spanish language sentence pairs), and a 0.72 correlation results with the Arabic sentence pairs as well Additionally, an extrinsic evaluation has been performed to investigate the performance and impact of the proposed approaches on three selected NLP applications, automatic short answer grading, paraphrase identification, and recognizing textual entailment. The simplified direct application of the proposed STS approaches in the selected applications did not achieve outstanding performance, but close to the state-of-the-art, and this will be considered as guidance for further improvements in the future.