Line Graph Enhanced AMR -to-Text Generation with Mix-Order Graph Attention Networks (2020)

Abstract: Efficient structure encoding for graphs with labeled edges is an important yet challenging point in many graph-based models. This work focuses on AMR-to-text generation – A graph-to-sequence task aiming to recover natural language from Abstract Meaning Representations (AMR). Existing graph-to-sequence approaches generally utilize graph neural networks as their encoders, which have two limitations: 1) The message propagation process in AMR graphs is only guided by the first-order adjacency information. 2) The relationships between labeled edges are not fully considered. In this work, we propose a novel graph encoding framework which can effectively explore the edge relations. We also adopt graph attention networks with higher-order neighborhood information to encode the rich structure in AMR graphs. Experiment results show that our approach obtains new state-of-the-art performance on English AMR benchmark datasets. The ablation analyses also demonstrate that both edge relations and higher-order information are beneficial to graph-to-sequence modeling.

## introduction abstract meaning representation @xcite is a sentence-level semantic representation formalized by a rooted directed graph, where nodes are concepts and edges are semantic relations. since amr is a highly structured meaning representation, it can promote many semantic related tasks such as machine translation @xcite and summarization @xcite . however, the usage of amr graphs can be challenging, since it is non-trivial to completely capture the rich structural information in the graph-based data, especially when the graph has labeled edges. to address the above limitations, we propose a novel graph-to-sequence model based on graph attention networks @xcite . we transform the edge labels into relation nodes and construct a new graph that directly reflects the edge relations. in graph theory, such a graph is called a line graph @xcite . as illustrated in figure 1 , we thus separate the original amr graph into two sub-graphs without labeled edges -concept graph and relation graph. the two graphs describe the dependencies of amr concepts and edges respectively, which is helpful in modeling these relationships (especially for edges). our model takes these sub-graphs as inputs, and the communications between the two graphs are based on the attention mechanism. furthermore, for both graphs, we mix the higher-order neighborhood information into the corresponding graph encoders in order to model the relationships between indirectly connected nodes. empirical study on two english benchmark datasets shows that our model reaches state-of-theart performance with 30.58 and 32.46 bleu scores on ldc2015e86 and ldc2017t10, respectively. in summary, our contributions include: ## mix-order graph attention networks in this section, we first introduce graph attention networks (gats) and their mix-order extensions, which are the basis of our proposed model. ## method the architecture of our method is illustrated in figure 3. as mentioned above, we separate the amr graph into two sub-graphs without labeled edges. our model follows the encoder-decoder architecture, where the encoder takes the two sub-graphs as inputs, and the decoder generates corresponding text from the encoded information. we first give some detailed explanations about the line graph and input representation. ## analysis as mentioned above, our system has two critical points: higher-order graph neighborhood information and relationships between amr edges. to verify the effectiveness of these two settings, we conduct a series of ablation tests based on different characteristics of graphs. ## conclusion and future work in this work, we presented a novel graph-tosequence approach which uses line graph to model the relationships between labeled edges from the original amr graph. the mix-order graph attention networks are found effective when handling indirectly connected nodes. the ablation studies also demonstrate that exploring edge relations brings benefits to graph-to-sequence modeling. furthermore, our framework can be efficiently applied to other graph-to-sequence tasks such as webnlg @xcite and syntax-based neural machine translation @xcite . in future work we would like to do several experiments on other related tasks to test the versatility of our framework. also, we plan to use large-scale unlabeled data to improve the performance further.