Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Review:
The authors address methods to encourage the emergence of the layout expression structures on the frameworks of neural module networks (NMN) for the visual QA problems. The methods are motivated from the works on language emergence for communication between multi-agents and the language acquisition of new-born babies from parents, which achieved with limited data. The methods, ‘iterative learning’ (IL) are designed as forming two agents (program generators and execution engines) to play VQA games. Basic architectures and learning methods seem to be very similar to the approach of semi-supervised learning introduced in [ICCV17]. 
This paper deals with one of very interesting topics, the language emergence among cooperative multi-agent environments and the compositionality of human language as recent related studies are well-surveyed in related work. In particular, the main idea of problem formation, layout expressions in NMN as emergent languages is very fresh and interesting.
The main claims are as follows: (1) the proposed approach of IL improves generalization performance for visual QA, and it is shown experimentally by comparing the ablation results of IL. (2) the language structures in the ground-truth data are recovered with only limited supervisions and the superiority is validated on two datasets – SHAPES-SyGeT and CLOSURE. However, I believe that the evidences for their claims are insufficient. Specifically, the authors do not provide enough information of language structure such as the superiority compared to other methods and the structure similarity of recovery levels.
I recommend 'ok, but not good enough – reject’ for this paper. 


Pros:
- The authors propose novel interesting problem and their solutions. Arguably, it seems potentially to be on one of important research flows to make influence to lots of works for academia in the future.
- They find and report good performance for out-of-distribution accuracies for visual QA datasets.

Concerns:
- It is not clear which parts in the proposed methods are novel with respect to previous works. Those make vague which parts are the authors’ contributions.
- I think IL should be clarified from semi-supervised learning approaches on them for visual QA. Also, for reproducibility, it should be specified which parts are with/without IL. What is ‘learning bottleneck’ on this approach? Also, it is not enough how program generators and execution engines are specified, even though some explanations are in appendix. I think it needs more links for reference or explanation.
- As mentioned above, the supports for main claims are not appropriate or unclear. It needs to theoretically or experimentally show the results of comparison with other methods and the similarity of the recovery level of language structures.
- Table 1 reports the result of comparative methods such as MAC and FiLM without program supervision. How are they configured in the experiments?
- I think that it would be better understandable to show usability and superiority with the experimental results on realistic visual QA such as VQA and GQA.

Minors:
- In Section 3.1,  γ. -> γ,  β. -> β
- It needs the description for operators and symbols for the formulae in Section 3.1

[ICCV17] Johnson et al., Inferring and executing programs for visual reasoning, ICCV 2017.

------------------------------------------------------------------------------------------------------------------------------
After rebuttal

From the revised version of this manuscript, the authors resolve my major concerns such as clarity/reproducibility of the method, differentiation from the previous works including semi-supervised learning, and scalability. So, I've raised my score to 6. 
Thank you for the contributions!

review_title: Revised review

final_score: rating: 7: Good paper, accept
review_content: This paper proposes to combine iterated learning (the process of repeated language transmission from a ‘parent’ agent to a ‘child’ agent) with neural module networks (NMNs), in order to emerge NMN layouts that perform better at systematic generalization. The paper evaluates on their new variant of the SHAPES dataset that tests systematic generalization, and on CLEVR / CLOSURE, showing improved systematic generalization performance while requiring a small amount of ground-truth layout supervision.

Pros:
- I think the idea of combining IL with NMNs is really clever (heh). Treating the program generator and execution engine as two agents that need to coordinate through a shared language (NMN layouts) is really interesting. If it were to work without layout supervision, it could open up new doors to applying IL + NMNs to many other tasks

- The paper is quite well-written, and easy to follow

- The related work section is thorough

- The experiments on SHAPES-SyGeT, show that IL helps significantly for generalization of NMN models 

- I appreciate the ablations in the Appendix. 


Cons: 
- One of the main questions I have about this approach is whether it will provide any benefit on more complex problems (e.g. large-scale VQA). There are a few reasons to think it might not be able to do so:
1) As alluded to in the paper, the IL procedure requires a lot of compute, which could be used to train larger models on more pre-training data
2) The improvement in validation performance on the more complex CLEVR dataset is fairly modest (though the program accuracy increase is large). While CLEVR is more complex than SHAPES, it is still a fairly artificial dataset targeted ‘compositional’ in nature, compared to general VQA. This suggests that it might be hard to get IL+NMN to work well on harder problems.
3) The method still requires (a small amount of) ground-truth layout supervision, which is not obtainable in general VQA or most other tasks.

- I would also like to have seen a bit more analysis / description of why the method currently fails without any ground-truth layout supervision. I think this would improve the paper a lot, as it would help other researchers improve upon the method to address this problem.



Overall:
I think the ideas in this paper are interesting enough, and the execution good enough, to warrant acceptance. While I have some concerns about whether this approach will scale, these questions will have to be answered in subsequent works and with further research.


Small typos:
“each new-born child need” -> needs

“Recently machine learning community also show” -> the machine learning community also shows

“Minimzing” -> minimizing 

review_title: Review

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The authors apply iterated learning - a procedure originating in CogSci analyses of how human languages might develop - to the training of neural module networks. The goal is for iterated learning to encourage these networks to develop compositional structures that support systematic generalization without requiring explicit pressures for compositional structures (in past work, such explicit pressures have generally been necessary). The proposed approach brings substantial improvements in systematic generalization across two datasets, SHAPES and CLEVR.

Strengths:

1. The approach is well-motivated, including impressive coverage of prior literature in both ML and CogSci.

2. The approach brings impressive gains in an area that is one of the major weaknesses of current ML systems, namely systematic generalization. 

3. In addition to the gains in accuracy, one particularly impressive benefit of this approach is the decreased amount of supervision that it requires compared to past approaches.

4. The paper is generally well-written and easy to follow. 

Weaknesses:

1. One of the motivations is to expand the use of iterated learning beyond toy datasets. While SHAPES and CLEVR may be not as toy-ish as datasets used in the past, they still are pretty toy-ish, so I’m not sure if this paper can reasonably claim that one of its contributions is to expand iterated learning to realistic domains.

2. Though the paper in general was very clear, I found Section 3.2 to be a bit hard to follow, and that section is important as it is the part that describes the structure of the iterated learning framing. I think this section would benefit from starting each subpart with a more high-level, intuitive description of what that stage accomplished, before diving into the details.

Minor comments:

1. Fodor et al only has 2 authors - Fodor and Pylyshyn

2. Page 3: “Although, the Gumbel straight-through estimator”: this use of “although” is usually frowned upon - better to use “However”

3. Page 5: typo: “minimzing”

4. In general, for the bibliography, check to see if a paper has been published at a conference or journal; if so, cite that version instead of the arXiv version. E.g., “Neural machine translation by jointly learning to align and translate.” was published in ICLR 2015, and “Systematic generalization: what is required and can it be learned?” was published at ICLR 2019.

review_title: Interesting and impressive application of iterated learning to VQA

Meta-review: This paper presents an original perspective on how to learn layouts and modules of neural module networks jointly, in a way that encourages the emergence of compositional solutions. In particular, layouts are treated as messages from an emergent language, and iterated learning is used to encourage the emergence of structure. The paper shows good performance in inducing compositional structure in two datasets.

Summarizing the reviewers' doubts, one is that the idea is tested on relatively toyish data sets, and it is not clear how it would scale up. The second, coming from one reviewer, concerns a lack of originality that, honestly, I do not see. If anything, this is probably the most original paper in my pool.

Concerning the first point, that is a fair objection, but I think that getting good results on program learning on datasets such as CLEVER is more than encouraging for a paper that is introducing quite a novel idea for the first time.

Finally, the authors added new text and new experiments strenghtening their conclusion during the discussion.

I am strongly in favour of accepting this paper.

Decision: Accept (Oral)