Optimal Partial Transport Based Sentence Selection for Long-form Document Matching (2022)

Abstract: One typical approach to long-form document matching is first conducting alignment between cross-document sentence pairs, and then aggregating all of the sentence-level matching signals. However, this approach could be problematic because the alignment between documents is partial — despite two documents as a whole are well-matched, most of the sentences could still be dissimilar. Those dissimilar sentences lead to spurious sentence-level matching signals which may overwhelm the real ones, increasing the difficulties of learning the matching function. Therefore, accurately selecting the key sentences for document matching is becoming a challenging issue. To address the issue, we propose a novel matching approach that equips existing document matching models with an Optimal Partial Transport (OPT) based component, namely OPT-Match, which selects the sentences that play a major role in matching. Enjoying the partial transport properties of OPT, the selected key sentences can not only effectively enhance the matching accuracy, but also be explained as the rationales for the matching results. Extensive experiments on four publicly available datasets demonstrated that existing methods equipped with OPT-Match consistently outperformed the corresponding underlying methods. Evaluations also showed that the key sentences selected by OPT-Match were consistent with human-provided rationales.

## introduction long-form document matching, which identifies the semantic relationship between a source document and a target document, has become a fundamental problem in both nlp and ir. representative tasks include cite recommendation @xcite and plagiarism detection @xcite etc. for example, in cite recommendation, * corresponding author document 2: medicinal and aromatic plants are nowadays becoming the main constituents for medicines, colorings, preservatives, and fragrances… although it is of high priority, the effect of slowgrowth conservation on the chemical profile of prestored medicinal plants after conservation was not extensively investigated .this research aimed to analyze in vitro stored thymbra spicata l. var. … document 1: ... the attraction of medicinal and aromatic plants as worthy farm crops has grown due to the demand created by consumer interest in these plants for culinary, medicinal, and other anthropogenic applications … an understanding of future opportunities in the medicinal and aromatic plant industry who 2003 is necessary to enable us growers to envision and invest in medicinal and aromatic crops that will meet market demands. figure 1 : a pair of matched long-form documents from s2orc dataset. documents 1 focuses on the future opportunities in the medicinal and aromatic plant industry. document 2 studies the vitro storage of spicata. though most sentences are not similar, document 1 cites document 2 (matched) because they both take medicinal and aromatic plants as examples (the highlighted sentences). the document matching has been used to recommend existing papers to be cited in a new paper. in plagiarism detection, the document matching model has been used to determine whether a paper is plagiarized from another paper. existing approaches to long-form document matching typically follow the paradigms developed for short-text (sentence) matching, i.e., they conduct matching based on all of the sentences in the documents. for example, @xcite ; @xcite ; @xcite map the documents into a latent semantic space from a hierarchical perspective (e.g., word, sentence, paragraph) and conduct matching in the semantic space. however, these methods ignore the fact that a long-form document usually contains multiple paragraphs and sentences, which convey complex and diverse semantics. unlike short-text matching where almost every word-level matching signal matters, in longform document matching, the alignment between a document pair is partial and a few matching signals between the key sentences can determine the matching result at the document level. figure 1 illustrates a typical example: two matched paper abstracts (document 1 cites document 2) from the semantic scholar open research corpus (s2orc) @xcite . these two abstracts are matched because of the highly matched signals between the highlighted key sentences. other parts of the abstracts are not important for matching. this observation inspires us that it is unnecessary to force a matching model to aggregate all of the sentence pairs. more importantly, the introduced noise may overwhelm the matching signals between key sentences. thus, identifying key sentences is becoming an essential step for document matching. in this paper, we propose a novel optimal partial transport (opt) @xcite based sentence selection component for existing long-form document matching model, namely opt-match. as illustrated in figure 2 , opt-match models the document alignment as an opt process by regarding two input documents (two sets of sentences) as two piles of earth. to achieve partial alignment, opt-match poses a limitation on how much the earth from one pile (document 1) needs to be transported to the other pile (document 2 ). based on the cost matrix whose elements are defined as inverse (or negative) similarities between sentences from different documents, opt-match generates an optimal transport plan (matrix) @xcite which indicates the alignment of sentences. therefore, key sentences can be extracted based on the transport plan. to make opt-match easily incorporated into existing document matching models, we provide two strategies to achieve the sentence selection. compared to existing ot-based methods @xcite @xcite , opt-match offers the following advantages in modeling the long-form document alignment: 1) opt-match models the partial nature of document alignment explicitly and flexibly, through limiting how much the mass to be transported; 2) opt-match allows the source and the target domains are not necessarily with the same total mass, which fits well with the phenomenon that the lengths of two documents may vary greatly. the ot-based methods, however, cannot take this into consideration; 3) opt-match is a model-agnostic approach, it can be easily plugged into a wide range of document matching models. to summarize, this paper makes the following main contributions: (1) we highlight the critical importance of the key sentence selection in long-form document matching, which has not yet been thoroughly studied in existing models. we conducted extensive empirical studies on four large-scale publicly available datasets. the experimental results demonstrated that opt-match improved existing document matching models and the sentences selected by opt-match were consistent with human rationales. ## conclusion in this paper, we highlight the critical role of conducting partial alignment in long-form document matching. a novel key sentence selection component based on optimal partial transport is proposed, called opt-match. opt-match automatically selects key sentences for document matching, addressing the issue that not every sentence in one document can correspond to a sentence in another document. moreover, opt-match can be easily incorporated into existing document matching models. comprehensive experiments on four public datasets show that opt-match consistently outperformed its underlying document matching models. the empirical analysis also verifies that the sentences selected by opt-match are not only consistent with human-provided rationales but also contributed to document matching.