Decomposing and Comparing Meaning Relations: Paraphrasing, Textual Entailment, Contradiction, and Specificity (2020)

Abstract: In this paper, we present a methodology for decomposing and comparing multiple meaning relations (paraphrasing, textual entailment, contradiction, and specificity). The methodology includes SHARel - a new typology that consists of 26 linguistic and 8 reason-based categories. We use the typology to annotate a corpus of 520 sentence pairs in English and we demonstrate that unlike previous typologies, SHARel can be applied to all relations of interest with a high inter-annotator agreement. We analyze and compare the frequency and distribution of the linguistic and reason-based phenomena involved in paraphrasing, textual entailment, contradiction, and specificity. This comparison allows for a much more in-depth analysis of the workings of the individual relations and the way they interact and compare with each other. We release all resources (typology, annotation guidelines, and annotated corpus) to the community.

## a all children receive the same education. in our annotation, we performed an additional annotation of the specificity and we identified the particular elements (words, phrases, clauses) in one sentence that were more specific than their counterpart. in example 9, we can identify that "girls" is more specific than "children". the difference in the specificity of "girls" and "children" is the reason why b is annotated as more specific than a. we called that "scope of specificity". in 80% of the pairs with specificity at textual level, our annotators were able to point at one or more particular elements that are responsible for the difference in specificity. in 20% of the pairs, the specificity was not decomposable. this finding also confirms @xcite 's findings, who showed that frequency-based features are well-suited for automatic specificity detection. in our analysis on the nature of the specificity relation, we combined the annotation of "scope of specificity" and the traditional annotation of linguistic and reason-based types discussed in the previous sections. in particular, we looked for overlap between the "scope of specificity" and the scope of linguistic and reason-based types. example 10 shows the two separate annotations side by side. in a and b, we show the annotation of the linguistic and reason-based types: "same polarity substitution (habitual)" of "children" and "girls", and "diathesis alternation" of "receive" and "is received by". in c and d we show the annotation of the specificity: "children" -"girls". when we compare the two annotations we can observe that the "scope of specificity" overlaps with the scope of "same polarity substitution (habitual)". we argue that when there is an overlap between the "scope of specificity" and a linguistic or a reason-based type, it is the linguistic or reason-based phenomenon that is responsible for the difference in specificity. in example 10 we can say that the substitution of "children" and "girls" is responsible for the difference of specificity.