Before revision:  We attribute the accuracy gaps to two disadvantages of non-autoregressive models : a) learning simultaneous generation under the overly strong conditional independence assumption; b) lacking explicit target language modeling. In this paper , we propose Glancing Transformer (GLAT) to address the above disadvantages, which reduces the difficulty of learning simultaneous generation and introduces explicit target language modeling in the non-autoregressive setting at the same time . Experiments on several benchmarks demonstrate that our approach significantly improves the accuracy of non-autoregressive models without sacrificing any inference efficiency .
After revision:  We attribute the accuracy gaps to two disadvantages of non-autoregressive models . Inspired by the way of learning word dependencies in autoregressive and iterative-decoding models , we propose Glancing Transformer (GLAT) to address the above disadvantages, which reduces the difficulty of learning simultaneous generation and introduces explicit target language modeling in the non-autoregressive setting at the same time . Experiments on several benchmarks demonstrate that our approach significantly improves the accuracy of non-autoregressive models without sacrificing any inference efficiency .