Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes a new kind of capsules for CNN. The capsule contains a 4x4 pose matrix motivated by 3D geometric transformations describing the relationship between the viewer and the object (parts). An EM-type of algorithm is used to compute the routing.

The authors use the smallNORB dataset as an example. Since the scenes are simulated from different viewer angles, the pose matrix quite fits the motivation. It would be more beneficial to know if this kind of capsules is limited to the motivation or is general. For example, the authors may consider reporting the results of the affNIST dataset where the digits undergo 2D affine transformations (in which case perhaps 3x3 pose matrices are enough?).

Minor: The arguments in line 5 of the procedure RM Routing(a,V) do not match those in line 1 of the procedure E-Step.

Section 2.1 (objective of EM) is unclear. The authors may want to explicitly write down the free energy function.

The section about robustness against adversarial attacks is interesting.

Overall the idea appears to be useful but needs more empirical validation (affNIST, ImageNet, etc).

review_title: Idea is interesting; need more empirical validation than smallNORB

final_score: rating: 7: Good paper, accept
review_content: The paper proposes a novel architecture for capsule networks. Each capsule has a logistic unit representing the presence of an entity plus a 4x4 pose matrix representing the entity/viewer relationship. This new representation comes with a novel iterative routing scheme, based on the EM algorithm.
Evaluated on the SmallNORB dataset, the approach proves to be more accurate than previous work (beating also the recently proposed "routing-by-agreement" approach for capsule networks by Sabour et al.). It also generalizes well to new, unseen viewpoints and proves to be more robust to adversarial examples than traditional CNNs.

Capsule networks have recently gained attention from the community. The paper addresses important shortcomings exhibited by previous work (Sabour et al.), introducing a series of valuable technical novelties.
There are, however, some weaknesses. The proposed routing scheme is quite complex (involving an EM-based step at each layer); it's not fully clear how efficiently it can be performed / how scalable it is. Evaluation is performed on a small dataset for shape recognition; as noted in Sec. 6, the approach will need to be tested on larger, more challenging datasets. Clarity could be improved in some parts of the paper (e.g.: Sec. 1.1 may not be fully clear if the reader is not already familiar with (Sabour et al., 2017); the authors could give a better intuition about what is kept and what is discarded, and why, from that approach. Sec. 2: the sentence "this is incorrect because the transformation matrix..." could be elaborated more. V_{ih} in eq. 1 is defined only a few lines below; perhaps, defining the variables before the equations could improve clarity. Sec. 2.1 could be accompanied by mathematical formulation).
All in all, the paper brings an original contribution and will encourage further research / discussion on an important research question (how to effectively leverage knowledge about the part-whole relationships).

Other notes:
- There are a few typos (e.g. Sec. 1.2 "(Jaderberg et al. (2015)",  Sec. 2 "the the transformation", Sec. 4 "cetral crop" etc.).
- The authors could discuss in more detail why the approach does not show significant improvement on NORB with respect to the state of the art.
- The authors could provide more insights about why capsule gradients are smaller than CNN ones.
- It would be interesting to discuss how the network could potentially be adapted, in the future, to: 1. be more efficient 2. take into account other changes produced by viewpoint changes (pixel intensities, as noted in Sec. 1).
- In Sec, 4, the authors could provide more details about the network training.
- In Procedure 1, for indexing tensors and matrices it might be better to use a comma to separate dimensions (e.g. V_{:,c,:} instead of V_{:c:}).
review_title: A novel approach for capsule networks

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper describes another instantiation of "capsules" which attempt to learn part-whole relationships and the geometric pose transformations between them.  Results are presented on the smallNORB test set obtaining impressive performance.

Although I like very much this overall approach, this particular paper is so opaquely written that it is difficult to understand exactly what was done and how the network works.  It sounds like the main innovation here is using a 4x4 matrix for the pose parameters, and an iterative EM algorithm to find the correspondence between capsules (routing by agreement).  But what exactly the pose matrix represents, and how they get transformed from one layer to the next, is left almost entirely to the reader's imagination.  In addition, how EM factors in, what the probabilities P_ih represent, etc. is not clear.  I think the authors could do a much better job explaining this model, the rationale behind it, and how it works.

Perhaps the most interesting and compelling result is Figure 2, which shows how ambiguity in object class assignment is resolved with each iteration.  This is very intriguing, but it would be great to understand what is going on and how this is happening.

Although the results are impressive, if one can't understand how this was achieved it is hard to know what to make of it.


review_title: An extremely opaque paper with a potentially interesting idea and good results

Meta-review: Authors present a new multi-layered capsule network architecture, implemented an EM routing procedure, and introduced "Coordinate Addition".  Capsule architectures are gaining interest because of their ability to achieve equivariance of parts, and employ a new form of pooling called "routing" (as opposed to max pooling) which groups parts that make similar predictions of the whole to which they belong, rather than relying on spatial co-locality. New state-of-art performances are being achieved on focused datasets, for which the authors have continued the trend.

Pros:
- New significant improvement to state-of-art performance is obtained on smallNORB, both in comparison to CNN structure as well as the most recent previous implementation of capsule network.

Cons:
- Some concern arose regarding the writing of the paper and the ability to understand the material, which authors have made an effort to address.

Given the general consensus of the reviewers that this work should be accepted, the general applicability of the technology to multiple domains, and the potential impact that improvements to capsule networks may have on an early field, area chair recommends this work be accepted as a poster presentation. 
Decision: Accept (Poster)