Non-Autoregressive Sentence Ordering (2023)

Abstract: Existing sentence ordering approaches generally employ encoder-decoder frameworks with the pointer net to recover the coherence by recurrently predicting each sentence step-by-step. Such an autoregressive manner only leverages unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences for ordering. To overcome these limitations, in this paper, we propose a novel Non-Autoregressive Ordering Network, dubbed NAON , which explores bilateral dependencies between sentences and predicts the sentence for each position in parallel. We claim that the non-autoregressive manner is not just applicable but also particularly suitable to the sentence ordering task because of two peculiar characteristics of the task: 1) each generation target is in deterministic length, and 2) the sentences and positions should match exclusively. Furthermore, to address the repetition issue of the naive non-autoregressive Transformer, we introduce an exclusive loss to constrain the exclusiveness between positions and sentences. To verify the effectiveness of the proposed model, we conduct extensive experiments on several common-used datasets and the experimental results show that our method outperforms all the autoregressive approaches and yields competitive performance compared with the state-of-the-arts. The codes are available at: https://github.com/steven640pixel/nonautoregressive-sentence-ordering .

## introduction sentence ordering is one of the fundamental and common tasks to model document coherence, which targets at re-organizing a set of sentences into a coherent paragraph (as shown in figure 1 ). most early works of sentence ordering @xcite @xcite apply probabilistic transition model and rule-based model, e.g., hmm, entity, and content model, based on hand-crafted features. while such unordered sentences s1. she poured herself a glass. ## experiments to verify the effectiveness of our naon, we conduct extensive experiments on six datasets, and evaluate the performance with acc, pmr, and τ . more detailed experimental settings are described in the appendix b. ## accelerating sentence ordering non-autoregressive strategy implementing parallel decoding is with low latency and is more efficient than the autoregressive one @xcite . in this part, we investigate the time efficiency of our naon empirically. we compare the inference time of naon and aon, which share the same encoder and employ different decoders, i.e., nonautoregressive and autoregressive decoders. ## conclusion we presented a novel non-autoregressive ordering network for sentence ordering, which consists of a basic sentence encoder, a contextual sentence encoder, and a non-autoregressive decoder. different from existing autoregressive methods, the proposed naon predicted sentence for each position in parallel, which also leveraged bilateral dependencies among sentences, and demonstrated competitive performance. to alleviate the repetition problem of the proposed naon, we devised an exclusive loss to constrain the exclusiveness matching between sentences and positions. the experimental results on several common-used datasets demonstrated the effectiveness of the proposed approach. the analyses of exclusive loss also indicate that alleviating repetition issue could improve the generation performance of the non-autoregressive decoder.