DIFM :An effective deep interaction and fusion model for sentence matching (2022)

Abstract: “Natural language sentence matching is the task of comparing two sentences and identifying the relationship between them. It has a wide range of applications in natural language processing tasks such as reading comprehension, question and answer systems. The main approach is to compute the interaction between text representations and sentence pairs through an attention mechanism, which can extract the semantic information between sentence pairs well. However, this kind of methods fail to capture deep semantic information and effectively fuse the semantic information of the sentence. To solve this problem, we propose a sentence matching method based on deep interaction and fusion. We first use pre-trained word vectors Glove and characterlevel word vectors to obtain word embedding representations of the two sentences. In the encoding layer, we use bidirectional LSTM to encode the sentence pairs. In the interaction layer, we initially fuse the information of the sentence pairs to obtain low-level semantic information; at the same time, we use the bi-directional attention in the machine reading comprehension model and self-attention to obtain the high-level semantic information. We use a heuristic fusion function to fuse the low-level semantic information and the high-level semantic information to obtain the final semantic information, and finally we use the convolutional neural network to predict the answer. We evaluate our model on two tasks: text implication recognition and paraphrase recognition. We conducted experiments on the SNLI datasets for the recognizing textual entailment task, the Quora dataset for the paraphrase recognition task. The experimental results show that the proposed algorithm can effectively fuse different semantic information that verify the effectiveness of the algorithm on sentence matching tasks.”

## introduction natural language sentence matching is the task of comparing two sentences and identifying the relationship between them. it is a fundamental technique for a variety of tasks. for example, in the paraphrase recognition task, it is used to determine whether two sentences are paraphrased. in the text implication recognition task, it is possible to determine whether a hypothetical sentence can be inferred from a predicate sentence. recognizing textual entailment (rte), proposed by dagan @xcite , is a study of the relationship between premises and assumptions. it mainly includes entailment, contradiction, and neutrality. the main methods for recognizing textual entailment include the following: similarity-based methods @xcite , rule-based methods @xcite , alignment feature-based machine learning methods @xcite , etc. however, these methods can't perform well in recognition because they didn't extract the semantic information of the sentences well. in recent years, deep learning-based methods have been effective in semantic modeling, achieving good results in many tasks in nlp @xcite @xcite . therefore, on the task of recognizing textual entailment, deep learning-based methods have outperformed earlier approaches and become the dominant recognizing textual entailment method. for example, @xcite recurrent neural networks to model premises and hypotheses, which have the advantage of making full use of syntactic information @xcite .after that, he first applied lstm sentence models to the rte domain by encoding premises and hypotheses through lstm to obtain sentence vectors @xcite . wang et al.proposed mlstm model on this basis, which focuses on splicing attention weights in the hidden states of the lstm, focusing on the part of the semantic match between the premise and the hypothesis.the experimental results showed that the method achieved good results on the snli dataset @xcite . paraphrase recognition is also called paraphrase detection.the task of paraphrase recognition is to determine whether two texts hold the same meaning.if they have the same meaning,they are called paraphrase pairs.traditional paraphrase recognition methods focus on text features.however,there are problems such as low accuracy rate.therefore,deep learning-based paraphrase recognition methods have become a hot research topic.deep learning-based paraphrase recognition methods are mainly divided into two types; 1) calculated word vectors by neural networks,and then calculated word vector distances to determine whether they were paraphrase pairs.for example, huang et al.used an improved emd method to calculate the semantic distance between vectors and obtain the interpretation relationship(dong-hong, 2017). 2) directly determining whether a text pair is a paraphrased pair by a neural network model,which is essentially a binary classification algorithm.wang et al.proposed the bimpm model, which first encodes sentence pairs by a bidirectional lstm and then matches the encoding results from multiple perspectives in both directions @xcite .chen et al.proposed an esim model that uses a twolayer bidirectional lstm and a self-attention mechanism for encoding, then it extracts features through the average pooling layer and the maximum pooling layer, and finally performs classification @xcite . these models mentioned above have achieved good results on specific tasks, but most of these models have difficulty extracting deep semantic information and effectively fusing the extracted semantic information,in this paper, we propose a sentence matching model based on deep interaction and fusion .we use the bi-directional attention and self-attention to obtain the high-level semantic information.then,we use a heuristic fusion function to fuse the low-level semantic information and the high-level semantic information to obtain the final semantic information.we conducted experiments on the snli datasets for the recognizing textual entailment task, the quora dataset for the paraphrase recognition task.,the results showed that the accuracy of the proposed algorithm on the snli test set is 87.1%, and the accuracy of the quora test set is 86.8%. our contributions can be summarized as follows: ## bidaf model based on bi-directional attention flow in the task of extractive machine reading comprehension, seo et al.first proposed a bi-directional attention flow model bidaf (bi-directional attention flow) for question-to-article and article-toquestion @xcite . its structure is shown in figure 1 . the model mainly consists of an embed layer, a contextual encoder layer, an attention flow layer, a modeling layer, and an output layer. after the character-level word embedding and the pre-trained word vector glove word embedding, the contextual representations x and y of the article and the question are obtained by a bidirectional lstm, respectively.the bi-directional attention flow between them is computed, and it proceeds as follows: we builds on this work by looking at sentence pairs in a natural language sentence matching task as articles and problems for reading comprehension.we use the bi-directional attention and self-attention to obtain the high-level semantic information.then,we use a heuristic fusion function to fuse the low-level semantic information and the high-level semantic information to obtain the final semantic information. ## method in this section, we describe our model in detail. as shown in figure 2 , our model mainly consists of an embedding layer, a contextual encoder layer, an interaction layer, a fusion layer, and an output layer.