Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper designed a new generative model by capturing the auto-regressive order as latent variables for sequence generation task. Based on combinatorical optimization techniques, the authors derived an policy gradient algorithm to optimize the variational lower bound. Empirical results on image caption and code generation showed that this method is superior than both fixed-order generation and previous adaptive-order method transformer-InDIGO. The authors further analyzed the learned orders on global and local level on COCO2017 dataset, demonstrating that the arrangement tend to follows the best-first strategy.


Concerns:
1. effect of sample size K: In section 5 training part, the paper claimed "For our model trained with Variational Order Inference , we sample K = 4 latents for each
training sample.". The sample size K is used to approximate the gradient in variational order inference and it also affects the training efficiency i.e. $O(NKdl^2)$. It's not clear how the author choose the appropriate sample size K. Some analysis or experiemnt reults on the sensitivity of sample size K will help clarify this concern.


2. experiments on nmt & running time: The papers didn't report any results on machine translation, an important task on conditional sequence generation. Since previous work(e.g. transformer-InDIGO) demonstrated superior results on several translation datasets, it's recommended that the authors also showed results on these datasets. Also one strength of the approach is its potential of fully parallelizing. A running time comparison will provide more convincing evidence to this claim.


3. figure: Figure 1. in section 4 showed the structure of variational order inference. Considering the paper is mainly focused on conditional generation, an conditional generation version will be better by incorporating x sequence into the figure.
review_title: Interesting idea on modeling generation orders with latent variables

final_score: rating: 7: Good paper, accept
review_content: - Summary
This paper aims to decode both content and ordering of language models and proposes Variational Order Inference (VOI). The authors introduce a latent sequence variable z = (z_1, .. ,z_n) in which z_t is defined as the absolute position of the value generated. The authors model the posterior distribution of z as a Gumbel-Matching distribution which is relaxed as a Gumbel-Sinkorn distribution. To training the encoder and decoder networks, the ELBO is maximized using the policy gradient with baseline. The experimental results on Django and MS-COCO 2017 dataset show the proposed VOI outperforms the Transformer-InDIGO, as well as suggests that learned orders depend on content and best-first generation order.


- Strong points
	1. The research on non-autoregressive orders to generate language is interesting, and the proposed method using Gumbel-Sinkorn distribution is mathematically well sound and novel.
	2. The proposed method outperforms the previous Transformer-InDIGO and other baselines (Random, L2R, Common, Rare). This paper analyzed the learned orders globally and locally, and conducted ablation studies.
	3. The paper is well-written and the authors also provide source codes for reproducibility.

- Weak points
	1. The results are not compared with other SOTA auto-regressive algorithms.

- Questions
	- How do you think about transferring knowledge from auto-regressively trained such as GPT-2 to such non-autoregressive models? Using the pertained model for the encoder and decoder would improve the results? Have you tried this strategy?
review_title: Interesting problem,  proposed methods, and experimental results.

final_score: rating: 7: Good paper, accept
review_content: The authors  propose the first domain-independent unsupervised learner that discovers high-quality autoregressive orders through fully-parallelizable end-to-end training without domain specific tuning.  Inspired by the variational auto-encoder, they propose an
encoder architecture to infer autoregressive orders. To solve  the non-differentiable ELBO ( discrete latent variables), they further construct a  practical algorithm with the help of policy gradients. The experiment results, such as the  global and local statistics for learned orders, are convincing.

Some problems.
1: The authors should clearly define the two similarity metrics between autoregressive orders in the appendix.
2: Please check up the X_axis in the Fig 3.
3: It would be better if the authors provide more results in the appendix,  such as the ablation studies about the 'K' and visualizations of sequences generated by the baselines.


review_title: official review by AnonReviewer4

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes to model the generation order as latent variables for sequence generation tasks, by optimizing the ELBO involving a proposed process of Variational Order Inference (VOI). To alleviate the difficulty of optimizing discrete latent variables, the authors propose to cast it as a one-step Markov Decision problem and optimize it using the policy gradient. The authors also introduce the recent developed Gumbel-matching techniques to derive the close-form of the posterior distribution.

Pros:
1. Overall, I think the research problem, i.e., explicit modeling the generation order,  in this work is interesting and worthy of discovering
2. Casting the optimization of discrete latent variables as a one-step MDP is interesting
3. Experiments show that the induced "best-first" order outperforms fixed orders, which verifies the motivation of the paper`
4. Extensive and inspiring analysis 

Cons:
1. Explicit modeling the generation order is not a very novel idea that there have been many works on this topic. 
2. For checking the generalization of the method and better comparison w/ InDIGO (though InDIGO also conducted on MSCOCO, Django and the current comparison is sufficiently fair), I would like to increase my rating if seeing more experiments on large scale machine translation benchmarks as those in InDIGO. This would also further support your claim of a general-purpose approach w/ little domain knowledge (if any).

-------
Minors:
- figure 1: could consider adding x, which would better match the descriptions of the paper (modeling p(y|x) instead of p(y))

Missing references: 

[1] Chan, W., Kitaev, N., Guu, K., Stern, M. and Uszkoreit, J., 2019. KERMIT: Generative insertion-based modeling for sequences. arXiv preprint arXiv:1906.01604.

[2] Gu, J., Wang, C. and Zhao, J., 2019. Levenshtein transformer. In Advances in Neural Information Processing Systems (pp. 11181-11191).

[3] Bao, Y., Zhou, H., Feng, J., Wang, M., Huang, S., Chen, J. and Li, L., 2019. Non-autoregressive transformer by position learning. arXiv preprint arXiv:1911.10677.

review_title: Review

Meta-review: This paper deals with a particular model structure selection problem: inferring the order of a given sequence of latent variables. This problem is closely related to the matching problem that involves discrete optimization. The authors propose to cast the problem into a one-step Markov Decision problem and optimize it using the policy gradient.  The proposal here is using Variational Order Inference (VOI) using and using a Gumbel-Sinkhorn distribution to construct a proposal over approximate permutations. The approach is mathematically sound and novel.

Empirical results on image caption and code generation show promising results: method outperforms the previous Transformer-InDIGO and other baselines (Random, L2R, Common, Rare). This paper further analyzes the learned orders globally and locally, and conducts ablations.

The reviewers were overall very enthusiastic. 

Decision: Accept (Poster)