Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: 
After reading reviews and comments I have decided to confirm the initial rating.

===================

The work presents an approach to encode latent representations of objects such that there are separate and disentangled representations for location and appearance of objects in a scene.  The work presents impressive qualitative results which shows the practical use of the proposal on multi-mnist and multi-dSprites. 

While the use of inference networks proposing positions for the network as a means of improving the disentanglement is clever and seems novel, though not unlike inference sub-networks which are well-known in conditional generation, the evaluation is not up to a standard I can endorse, resulting in a recommendation to reject. 

Despite the interesting qualitative results, I will have to quote the work in saying, “All methods cited here are likelihood based so they can and should be compared in terms of test log likelihood. We leave this for future work.”.  Indeed the cited works should have been evaluated against, especially Greff et al. 2019, Nash et al, 2017, and Eslami et al, 2016, which are all very similar.  As written it’s impossible to tell whether this work actually improves over the state of the art, we only have the constructed baseline (which as a community we all know clearly would not have worked). 

A figure showing the relevant submodules of the network architecture and what they do in relation to the overall method would be helpful to understand the pipeline and how the inference network relates to the whole.

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper presents a probabilistic generative model for identifying the location, type and colour of images placed within a grid. The paper is in general well written and presents a large number of visual images demonstrating that the approach works. The main concerns with the paper are as follows:

1) The implementation details for the work are relegated to an appendix. As this is a core concept for the work one would expect this to be presented in the main body of the work.

2) Although there are a large number of visual images there is little in the way of analytical analysis of the work. As a particular concern the authors claim that figure 3 ‘prove’ that objects are disentangled. From Maths 101 I remember it being drummed into us that ‘proof by example is not a proof’.

3) The are parts of the process which are not explained for example, why is the following conducted? ‘The digits are first rescaled from their original size (28 × 28) to 15 × 15 by bilinear interpolation, and finally binarized by rounding.’

4) I would also like to know why this is called disentangling as the whole process prevents the ‘icons’ from overlapping. Disentangle normally refers to things which are at least overlapping. There are examples of works in this area.

5) The example cases are simple. One would expect at least one real-world example.
review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: This paper introduces a compositional generative model of images, where the image is described by a variable number of latent variables. Moreover, the latent variables are disentangled, in the sense that they represent different parts of the scene, and where appearance and location are described separately. While the generative model is very similar to AIR [1], the central claim of the paper is that the proposed inference scheme can generalize to a much higher number of objects than seen during training, which is demonstrated empirically, and with which AIR struggles. Better generalization is achieved by removing the recurrent core of AIR and replacing it with a fully-convolutional inference network, which predicts appearance vectors at every location in the feature map. The appearance vectors are then stochastically sampled without replacement according to a location distribution. Unlike AIR, this approach does not model the object scale.

I recommend REJECTing this paper. While the improved generalization performance is a useful property, it has been achieved previously and in a very similar fashion by SPAIR [2], which follows a similar model design. SPAIR still uses an RNN, while the proposed approach does not, but this is only a small simplification and does not warrant publication at a top tier conference. There are no other contributions in this paper. Additionally, on the one hand, the experimental evaluation is insufficient: the proposed approach is compared only against a fully-convolutional VAE, while very similar models like AIR [1], SPAIR [2], SuPAIR [3] are not considered. On the other hand, the experimental section focuses on the disentanglement of representations, which is a) evident from the model construction and b) achieved in all previous models. 

The paper is clearly written, and the presented generative model is interesting. Having said that, the problem that this paper addresses is mostly solved in [2] and [4]; also both [2] and [4] scale the general approach introduced in [1] to much higher number of objects (in the hundreds) and more general settings (real images, atari games).

[1] Eslami et. al., “Attend, Infer, Repeat:...”, NIPS 2016.
[2] Crawford and Pineau, “Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks”, AAAI 2019.
[3] Stelzner et. al., “Faster Attend-Infer-Repeat with Tractable Probabilistic Models”, ICML 2019.
[4] Jiang et. al., “Scalable Object-Oriented Sequential Generative Models”, arXiv 2019.
review_title: Official Blind Review #1

Meta-review: This paper presents a VAE approach where the model learns representation while disentangling the location and appearance information. The reviewers found issues with the experimental evaluation of the paper, and have given many useful feedback. None of the reviewers were willing to change their score during the discussion period. with the current score, the paper does not make the cut for ICLR, and I recommend to reject this paper. 
Decision: Reject