Compositional Phrase Alignment and Beyond (2020)

Abstract: Phrase alignment is the basis for modelling sentence pair interactions, such as paraphrase and textual entailment recognition. Most phrase alignments are compositional processes such that an alignment of a phrase pair is constructed based on the alignments of their child phrases. Nonetheless, studies have revealed that non-compositional alignments involving long-distance phrase reordering are prevalent in practice. We address the phrase alignment problem by combining an unordered tree mapping algorithm and phrase representation modelling that explicitly embeds the similarity distribution in the sentences onto powerful contextualized representations. Experimental results demonstrate that our method effectively handles compositional and non-compositional global phrase alignments. Our method significantly outperforms that used in a previous study and achieves a performance competitive with that of experienced human annotators.

## introduction phrase alignment is a fundamental problem in modelling the interactions between a pair of sentences, such as paraphrase identification, textual entailment recognition, and question answering @xcite @xcite . phrase alignment generally adheres to compositionality, in which a phrase pair is aligned based on the alignments of their child phrases. nonetheless, non-compositional alignments involving long-distance phrase reordering are prevalent in practice @xcite @xcite . figure 1 shows an example of phrase alignment in which phrases of the same colours are alignable, i.e. they are phrasal paraphrases. the alignment of 'antivirus vaccines' and 'vaccines against the virus' is compositional, as supported by alignments of their child nodes although their orderings are reversed. the scientific team created a vaccin relying on team spirit , the research group antivirus developed vaccines the scientific team created vaccines against the virus through teamwork similarly, the alignment of their parents τ s 2 and τ t 2 is compositional. by contrast, the alignment of τ s 1 and τ t 1 is non-compositional in relation to the alignment of τ s 2 and τ t 2 ; although τ t 1 and τ t 2 are siblings, τ s 1 is not a sibling of τ s 2 , i.e. not in the scope of the parent node of τ s 2 . to treat such a long-distance correspondence in non-compositional alignment, one has to consider candidate phrases outside the local scope and potentially the entire sentence. in this study, we address the phrase alignment problem by combining a tree mapping algorithm with phrase representation modelling. we treat compositional alignment by an algorithm for an unordered tree mapping @xcite . for the algorithm to work, definition of the edit cost (i.e. dissimilarity between phrases) is crucial. we propose a novel phrase representation, by which the edit cost is defined, based on contextualized representations by the bidirectional encoder representations from transformers (bert) @xcite . the proposed phrase representation models the similarity distribution in the entire sentence, thereby allowing the algorithm to be extended to treat non-compositional global alignments. phrase alignment can be difficult even for humans because there is unavoidable subjectivity in acceptable semantic discrepancies between paraphrases. our experimental results indicate that the proposed method achieves 95.7% of the alignment quality of trained human annotators for phrase alignment in paraphrase sentence pairs. the contributions of this study are twofold. first, we formalise the compositional phrase alignment problem as an unordered tree mapping. second, we propose a phrase representation model that allows non-compositional global alignments. ## phrase representation for alignment we propose a phrase representation model on top of the pre-trained bert. one of the most common methods for obtaining a phrase representation from bert is pooling outputs corresponding to tokens in the phrase. however, as we empirically show in section 6, this method exhibits an unsatisfactory ability for modelling the similarity distribution in a sentence pair. hence, we propose a novel method for generating phrase representations suitable for the phrase alignment problem. ## discussion and future work in contrast to previous methods, ours can align phrases not only in paraphrasal sentence pairs but also in partially paraphrasal pairs. we plan to apply it to a comparable corpus of partial paraphrases and investigate the performance, with the aim of creating a large-scale syntactic and phrasal paraphrase dataset. we intend to expand our method to conduct forest alignments for making it robust against parsing errors, which are inevitable in handling large corpora. further, as our method does not restrict input to syntactic trees but only assumes tree structures with arbitrary numbering (e.g. leftto-right post-order numbering) as input, we intend to try alignments of chunk-based trees, which is desirable for applications that process text fragments, e.g. those that perform information extraction.