GraphSeq2Seq: Graph-Sequence-to-Sequence for Neural Machine Translation
Authors: Guoshuai Zhao, Jun Li, Lu Wang, Xueming Qian, Yun Fu
Keywords: Neural Machine Translation, Natural Language Generation, Graph Embedding, LSTM
TL;DR: Graph-Sequence-to-Sequence for Neural Machine Translation
Abstract: Sequence-to-Sequence  (Seq2Seq)  neural  models  have  become  popular  for  text generation problems,  e.g.   neural machine translation (NMT) (Bahdanau et al.,2014;  Britz  et  al.,  2017),  text  summarization  (Nallapati  et  al.,  2017;  Wang  &Ling, 2016), and image captioning (Venugopalan et al., 2015; Liu et al., 2017). Though sequential modeling has been shown to be effective, the dependency graph among words contains additional semantic information and thus can be utilized for sentence modeling. In this paper, we propose a Graph-Sequence-to-Sequence(GraphSeq2Seq) model to fuse the dependency graph among words into the traditional  Seq2Seq  framework.   For each sample,  the sub-graph  of each word is encoded to a graph representation, which is then utilized to sequential encoding. At last, a sequence decoder is leveraged for output generation. Since above model fuses different features by contacting them together to encode, we also propose a variant of our model that regards the graph representations as additional annotations in attention mechanism (Bahdanau et al., 2014) by separately encoding different features.  Experiments on several translation benchmarks show that our models can outperform existing state-of-the-art methods, demonstrating the effectiveness of the combination of Graph2Seq and Seq2Seq.