Paraphrase Identification via Textual Inference (2024)

Abstract: Paraphrase identification (PI) and natural language inference (NLI) are two important tasks in natural language processing. Despite their distinct objectives, an underlying connection exists, which has been notably under-explored in empirical investigations. We formalize the relationship between these semantic tasks and introduce a method for solving PI using an NLI system, including the adaptation of PI datasets for fine-tuning NLI models. Through extensive evaluations on six PI benchmarks, across both zero-shot and fine-tuned settings, we showcase the efficacy of NLI models for PI through our proposed reduction. Remarkably, our fine-tuning procedure enables NLI models to outperform dedicated PI models on PI datasets. In addition, our findings provide insights into the limitations of current PI benchmarks.

## introduction semantic relationships have been the subject of extensive research, and play pivotal roles in natural language processing @xcite @xcite @xcite , including the study and evaluation of the reasoning capabilities of language models @xcite . two important tasks that depend on semantic relations between sentences are paraphrase identification (pi; @xcite and natural language inference (nli; @xcite @xcite . pi is the task of deciding whether two sentences are in the paraphrase relation, that is, whether they convey the same meaning @xcite . nli involves three labels that describe the relationship between two sentences: entailment, contradiction, and neutral @xcite . our focus is specifically on detecting textual entailment, as indicated by the first of these categories ( @xcite ; poliak, 2020), or, more generally, textual inference @xcite , which is the relation between sentences where one can be inferred from the other in a given context. take the example from snli @xcite ; while the premise "this man is surfing" does not always entail the hypothesis "a man is on water", the broader context may make it clear that the word surfing refers to an aquatic activity rather than website browsing, and so the latter sentence can be inferred from the former. prior work has hypothesized that paraphrasing corresponds to bidirectional textual entailment; see, for example, the surveys of androutsopoulos and malakasiotis (2010) and @xcite . however, to the best of our knowledge, the only work that empirically investigates the connection between these two tasks is @xcite . they incorporate a blend of modules, including word sense disambiguation for sentence similarity and a markov logic network for probabilistic inference, which complicates the analysis of the interplay between paraphrases and entailment. moreover, their approach aligns more with traditional pi methods than with our approach, and lacks any theoretical formalization. in this work, we formalize prior informal observations on the relationship between textual entail-ment and paraphrasing into a coherent theoretical framework (figure 1 ). we formally define four semantic relations and classify them according to two criteria: symmetry and contextuality. this formalization implies a practical reduction of pi to nli, which we empirically validate by employing two widely used pre-trained transformer-based language models, roberta and xlnet. we introduce a dataset adaptation process for fine-tuning an nli model for pi, and test our implementation on six pi benchmarks. our results indicate that in the fine-tuned setting, our pi to nli reduction can actually yield better performance compared to the direct application of a pi system. this provides strong support for the utility of our reduction, and the theoretical model upon which it is based. ## methodology in this section, we present our theoretical framework linking four semantic relations. we also introduce a novel method for fine-tuning an nli model for pi, proposing a dataset adaptation procedure that converts pi datasets to labeled nli instances. ## experiments the experiments in this section are aimed at validating the proposed theoretical framework. additional data specifics and training details can be found in appendices b and c. ## conclusion we have presented a novel theoretical and empirical study of the relationship between two important semantic tasks, pi and nli, a topic that has remained largely unexplored. our experiments provide strong evidence that our innovative pi2nli reduction, combined with fine-tuning on the nli data facilitated by our dataset adaptation procedure, yields substantial f1 improvements on the pi task, outperforming dedicated pi models on benchmark pi datasets. the variable outcomes observed when applying pi2nli in a zero-shot setting also offer insights into the existing limitations of the current pi datasets. in addition to advancing the state of the art, our findings offer valuable insights into the relation between pi and nli, and set the stage for further investigation.