Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The authors propose a new artificial neural network architecture that is derived from a human visual model (Mély et al., 2018). The original (human vision) model can explain some of the human visual illusions, specifically contextual ones. While the adaptation from this human visual model to artificial neural networks was previously done by (Linsley et. al., 2018a), in this paper the authors extend (Linsley et. al., 2018a) to better capture some of the constraints in the human visual model, and also to add a formulation that can also model top-down connections (across layers).
The goal of replicating the structure of the visual human model in artificial neural networks is to improve the machine vision by mimicking the human vision. The results in this paper on contour detection show an improvement regarding sample efficiency with respect to other state of the art methods. Also, the authors show that the artificial model also suffers from visual illusions, and when these are explicitly corrected, its performance drops. This shows that these illusions are a byproduct of the system improving its visual abilities.

Strengths:
1 - Well motivated by cognitive science theory and modeling of the human visual system. The artificial modeling is not just inspired by the human visual system, but derived from an actual human system, replicating it. The paper presents an extension of (Linsley et. al., 2018a), and both the base model and the extension are biologically motivated. The paper also shows that this extension is important to get good results.
2 - Clear structure and ideas. Clearly explained, well written, reasonable ablations and transparent presentation of results, assumptions, limitations, contributions and experiments.
3 - Very good results in contour detection for a very-low data regime, which proves the strength of the model inductive bias. The results show both better performance and good mimicking of the human vision behavior. 

Weaknesses
1 – Limited experimental results. 
- Is contour detection the only task in which surround suppression helps?
- Lack of experiments in larger contour detection datasets (like Semantic Border Dataset or Cityscapes). Does the model improve accuracy in those, or it just improves sample efficiency in (extremely) small (subsets of) datasets? 
- The only results are regarding sample efficiency. Not accuracy, or number of parameters, or running time. 
2 – Weak results supporting the claim that the computer vision model mimics the human visual illusions. The authors show an elegant experiment where they explicitly correct the visual illusion and obtain worse results, backing the presented hypothesis. However, the ablation experiments show that the same model without the top-down connections does not present the same results. One would expect that removing a part that is not in the initial formulation from (Mély et al., 2018) should not affect in the visual illusion experiments. Both the explanation at the end of Section 2 and at the sixth paragraph of Section 3 seem to indicate that the _non-negativity_ is the important factor to explain contextual illusions, not the top-down formulation. Can the authors explain whether or not the top-down formulation is a necessary part to model (Mély et al., 2018)? If this is not the case (and top-down is not necessary), the current results would not support the idea that implementing (Mély et al., 2018) in artificial neural networks produces visual illusions in the computer. Also, did the authors perform the visual illusion experiment without the non-negativity? Is it a necessary requisite for the visual illusions to appear? As a positive remark regarding weakness #2, the results still show that the model that performs best is the one having visual illusions. 
3 – Related to weaknesses #1 and #2, did the authors perform any experiment on other contextual illusions like the ones explained in (Mély et al., 2018), namely “color induction” or “enhanced color shifts”? Consistent findings across different visual illusions would reinforce the presented hypothesis.

Additional comments:
- While it may not be a "bug", arguing that the visual illusions are a "feature" (both in machine and human visual systems) is probably too much of a claim. At some point the authors refer to it as a "byproduct" of the of the system improving its visual abilities, which I consider a more suitable word. 
- For an audience outside of neuroscience, a brief explanation of the concepts “suppression” and “facilitation”, which are very important in the paper, would be convenient.
- Are 8 time-steps sufficient for reaching a steady state? Is the steady state checked in any way?

-------- Updated --------
Rating:
- Weak accept
The authors addressed the initial concerns I raised, providing detailed explanations and additional experiments. While some of these concerns still remain to some extent, I believe this paper explores an interesting direction connecting human visual models with computer vision, obtaining good experimental results. The authors softened some of their initial claims, and the current ones are more supported by the experiments.
Overall, I believe this paper can be a valuable contribution to the conference, and I recommend acceptance.

-------- Previously --------
Rating:
- Weak reject 
Overall it is a clear and well-structured paper, with interesting biologically derived architectures (strength #1), but as it stands the experimental results either do not completely support the claims of the paper (weakness #2) or are limited in scope (weakness #1). If the authors can address my concerns, mostly motivating the importance of the experiments for weakness #1, and providing an explanation (possibly correcting me) for weakness #2, I will be happy to increase my rating. 
review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: The presented paper introduces a novel neural network architecture to explore the question whether visual illusions are corner cases of the human visual system, or whether they represent limitations of perception. The developed recurrent network architecture aims at being more sample efficient than existing methods. The findings discussed in the paper suggest that visual illusions are a byproduct of neural circuits that help to increase the robustness of the human visual system, which in turn suggests that neural networks for processing visual data could benefit from integrating circuits in similar ways. While existing work predominantly aims at explaining whether visual illusions are features or artifacts of the visual system, this work focuses on finding a computational solution to support the hypothesis that visual illusions are features. In particular, the contributions of this work are: (1) novel neural network architecture, called \gamma-networks, which is derived from the work of [Meley et al. 2018] and (2) that the proposed architecture is more sample efficient than SOTA convolutional architectures on contour detection tasks. 

Overall, I think this is an interesting paper that can be accepted for publication. However, I am not an expert in this field and am excited to see what the other reviewers think. While the overall contribution of this work appears rather narrow, exploring traits of the human visual system to leverage them in the context of convolutional neural networks for segmentation tasks appears interesting and has the potential to simulate further research in this field. Furthermore, the results are promising and show that the introduced approach is on par with SOTA work in the field. On the downside, many sections of the paper appear unnecessarily cryptic and lack important details. Other sections rely on explanations in the appendix or just refer to related work instead of providing context. This gives me the impression that this work might be better suited for a journal instead of trying to discuss all necessary details into the page limits of ICLR (the paper is also two pages over the common page count). Moreover the topic this paper addresses might be a better fit for a venue that specifically focuses on neuroscience (similar as Mely et al. 2018).   


Specific comments: 

- Abstract and introduction can be improved by more carefully introducing visual illusions and how the effects of specific traits of the human visual system lead to visual illusions. In its current form abstract and introduction are quite difficult to follow for readers that are less well-versed in neurscience. 
- While the main contributions are listed at the end of the introduction, they are not clearly described. It is not clear why it is beneficial that the network generates "an orientation-tilt illusion after it is optimized for contour detection". More details need to be provided here. 
- The term "hyper columns" is not properly introduced and needs better motivation, similarly for terms like "suppression" and "facilitation", "circuit integration". Overall it seems the methods section relies too much on the related work for explaining concepts and terminology, which makes this section quite difficult to follow. This needs to be improved. 
- It is not clear why it is beneficial to enforce non-negativity. The provided explanation is unclear: "The non-negativity constraints we introduce into the fGRU are necessary to guarantee separate stages of suppression followed by facilitation that can implement asymmetric contextual interactions", which is again because terms were not properly introduced. The term "fGRU" is introduced after it is used.
- Not providing training time with the argument that it is outside the scope of this work seems like a flawed argument and is uncommon practice.  
- The datasets listed in Section 4.1. are introduced without references. 
- The term "F1 ODS" is not properly introduced. 
- Figure 3 (a): the term "Ding" is not introduced. Does this refer to [Ding et al. 2016]?
- The references show many inconsistencies (abbreviated vs. long conference names, etc.). This should be fixed for the final version of this work. 
- No limitations are discussed. What are the edge cases in which the method does not perform well?


Typos: 
Sec 6: hihg-level
review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: The paper introduces a complex hierarchical recurrent model for contour detection loosely inspired by the organization of cortical circuits. Their model performs state-of-the-art on sample-limited versions of popular contour detection (BSDS500) and cell segmentation (SNEMI3D) datasets, and it reproduces the well-known tilt illusion when transfer-learning orientation estimation. Interestingly, "untraining" the tilt illusion degrades performance on contour detection.

Strengths:
+ State-of-the-art performance on data-limited contour detection tasks
+ Nice illustration of how the network refines its predictions over time
+ Demonstrates a contextual visual illusion in a task-trained neural network
+ Shows that tilt illusion is actually necessary for optimal performance in their model

Weaknesses:
- Architecture seems very complicated (unnecessarily so?)
- No ablation studies showing the usefulness of various model components
- Title seems a bit overly general given the quite specific result
- Not clear whether their results support their interpretation of the function of illusions

It's a relatively straightforward paper that is easy to follow and has a clear result that is both interesting and novel. Thus, I'm generally very supportive of the paper.

There are a couple of weaknesses summarised above and detailed below that I would love to see addressed, but none of them is overly critical:

1. Unfortunately the paper suffers from the same issue as the original work on fGRU, which it's based on: The fGRU architecture seems overly complicated and its numerous details and design choices not well motivated. Ablation studies showing which components are really necessary are missing. While this was understandable for the original paper, which introduced a novel approach, one would hope that follow-up work would subsequently get rid of some of the slack and simplify the architecture to the minimum that's really required.

2. The title suggests that the paper explains the function of contextual illusions in general, but the paper actually "just" shows that one contextual illusion emerges when one trains a biologically inspired model on one particular task. I suggest aligning the title better with the actual contribution.

3. (somewhat philosophical) The paper does not really answer the question posed in the abstract, does it? Do visual illusions reflect basic limitations of the visual system or do they correspond to corner cases of neural computations that are efficient in everyday settings? The authors seem argue for the second possibility. But if that was the case, wouldn't one expect other systems trained on the same tasks to also exhibit these illusions? It seems to me as if their results might suggest quite the opposite: Because only brain-like architectures exhibit this illusion, and because only they are hurt by "unlearning" the illusion, this visual illusion may reflect a basic limitation of how the visual system solves the task. I think it would be great if the author could comment on this point and clarify their reasoning in the paper.
review_title: Official Blind Review #1

Meta-review: All the reviewers recommend accept, and the found the paper interesting and novel. 
Decision: Accept (Poster)