  In this paper, we propose a unified explanation of representation for
layer-aware neural sequence encoders, which regards the representation as a
revisited multigraph called multi-order-graph (MoG), so that model encoding can
be viewed as a processing to capture all subgraphs in MoG. The relationship
reflected by Multi-order-graph, called $n$-order dependency, can present what
existing simple directed graph explanation cannot present. Our proposed MoG
explanation allows to precisely observe every step of the generation of
representation, put diverse relationship such as syntax into a unifiedly
depicted framework. Based on the proposed MoG explanation, we further propose a
graph-based self-attention network empowered Graph-Transformer by enhancing the
ability of capturing subgraph information over the current models.
Graph-Transformer accommodates different subgraphs into different groups, which
allows model to focus on salient subgraphs. Result of experiments on neural
machine translation tasks show that the MoG-inspired model can yield effective
performance improvement.
