Neural Graph Matching Networks for C hinese Short Text Matching (2020)

Abstract: Chinese short text matching usually employs word sequences rather than character sequences to get better performance. However, Chinese word segmentation can be erroneous, ambiguous or inconsistent, which consequently hurts the final matching performance. To address this problem, we propose neural graph matching networks, a novel sentence matching framework capable of dealing with multi-granular input information. Instead of a character sequence or a single word sequence, paired word lattices formed from multiple word segmentation hypotheses are used as input and the model learns a graph representation according to an attentive graph matching mechanism. Experiments on two Chinese datasets show that our models outperform the state-of-the-art short text matching models.

## introduction short text matching (stm) is a fundamental task of natural language processing (nlp). it is usually recognized as a paraphrase identification task or a sentence semantic matching task. given a pair of sentences, a matching model is to predict their semantic similarity. it is widely used in question answer systems and dialogue systems @xcite . the recent years have seen advances in deep learning methods for text matching @xcite @xcite . however, almost all of these models are initially proposed for english text matching. applying them for chinese text matching, we have two choices. one is to take chinese characters as the input of models. another is first to segment each sentence into words, and then to take these words as input tokens. although character-based models can overcome the * kai yu is the corresponding author. problem of data sparsity to some degree @xcite , the main drawback of these models is that explicit word information is not fully exploited, which can be potentially useful for semantic matching. however, word-based models often suffer some potential issues caused by word segmentation. as shown in figure 1 , the character sequence "南 京 市 长 江 大 桥(south capital city long river big bridge)" has two different meanings with different word segmentation. the first one refers to a bridge (segment-1, segment-2), and the other refers to a person (segment-3). the ambiguity may be eliminated with more context. additionally, the segmentation granularity of different tools is different. for example, "长江大 桥(yangtze river bridge)" in segment-1 is divided into two words "长江(yangtze river)" and "大 桥(bridge)" in segment-2. it has been shown that multi-granularity information is important for text matching @xcite . here we propose a neural graph matching method (gmn) for chinese short text matching. instead of segmenting each sentence into a word sequence, we keep all possible segmentation paths to form a word lattice graph, as shown in figure 1 . gmn takes a pair of word lattice graphs as input and updates the representations of nodes according to the graph matching attention mechanism. also, gmn can be combined with pre-trained language models, e.g. bert @xcite . it can be regarded as a method to integrate word information in these pre-trained language models during the fine-tuning phase. the experiments on two chinese datasets show that our model outperforms not only previous state-of-the-art models but also the pre-trained model bert as well as some variants of bert. ## problem statement first, we define the chinese short text matching task in a formal way. given two chinese sentences s @xmath0, c a 2 , • • • , c a ta } and in this paper, we propose a graph-based matching model. instead of segmenting each sentence into a word sequence, we keep all possible segmentation paths to form a word lattice graph @xmath1, e). v is the set of nodes and includes all character subsequences that match words in a lexicon d. e is the set of edges. if a node v @xmath2, then there is an edge e ij between them. n f w (v i ) denotes the set of all reachable nodes of node v i in its forward direction, while n bw (v i ) denotes the set of all reachable nodes of node v i in its backward direction. ## proposed framework as shown in figure 2 , our model consists of three components: a contextual node embedding module (bert), a graph matching module, and a relation classifier. ## conclusion in this paper, we propose a neural graph matching model for chinese short text matching. it takes a pair of word lattices as input instead of word or character sequences. the utilization of word lattice can provide more multi-granularity information and avoid the error propagation issue of word segmentation. additionally, our model and the pre-training model are complementary. it can be regarded as a flexible method to introduce word information into bert during the fine-tuning phase. the experimental results show that our model outperforms the state-of-the-art text matching models as well as some bert-based models.