Venue: MIDL 2018 Conference

Reviews:

final_score: rating: 3: Marginally above acceptance threshold
review_content: The main contribution of this paper is to propose an alternative semantic segmentation net, based on capsules and so, different from standard networks, while leading to competitive results.

The authors should rephrase some statements such as "The proposed algorithm fundamentally improves the current state-of-the-art object segmentation approaches, and provides strong evidence that capsules can successfully model the spatial relationships of the objects better than traditional CNNs". There is no evidence of this in the experimental section, which in itself presents some weaknesses:
First, because only one dataset is used. 
Second, because the differences reported in the performance look minimal between the proposed model and the competitors, and it is unclear if they are significant. 
Third, it is not clearly motivated why reconstruction is needed as a means for regularization. It is also unclear whether the competitors use the same regularization method or not. In any case, it would be desirable to have an experiment to check the effect of such regularization method.

The paper reads ok, although some sentences are difficult to parse (also there is a weird use of the exist symbol in Sec. 4).
In general, it is an interesting paper with promising results, but the empirical evidences are far from what it is claimed in the conclusions.
review_title: The authors extend capsule networks to the case of semantic segmentation, in an efficient way, by limiting the neighborhood of capsules that can be children of another one (also sharing transformation matrices).

final_score: rating: 4: Top 50% of accepted papers, clear accept
review_content: Authors present an extension of the capsule networks, which was originally published in 2017, for object segmentation. To the best of the knowledge of this reviewer, this is the first article that applies capsule networks to image segmentation. 
Pros: 
1. This is the first application of capsules to image segmentation. In that respect, the contribution is original and introduces an interesting avenue for image segmentation research. 
2. Authors propose a simple yet effective way of extending capsules for the segmentation problem without increasing the memory requirement and the number of parameters by a large margin. 
3. The explanation of the method somewhat relies on the original capsule network paper. However, despite that the method is clearly explained and the text is well written. 

Cons: 
1. The motivation for using capsules for object segmentation is not explicitly made. Authors mention the advantages of capsules for object detection but these are rewritten from the original work. 
2. The evaluation is the weakest spot of this article in my opinion. The following points reduces enthusiasm for the article: 
2a. Authors claim that the proposed structure reduces network parameters compared to U-Net and Tiramisu. This statement is rather vague. Authors did not perform any experiments with U-Net and Tiramisu using similar parameters as SegCaps. Segmentation performance of these networks with low number of parameters would be useful to back-up the claim of reducing required parameters. 
2b. There are two components of the proposed network that are hard to disentangle. First component is the capsules and the convolutional-deconvolutional extensions. Second component is the regularisation by reconstruction. The same regularisation used with the other networks might also yield similar accuracy improvements. 
2c. The improvements in Dice scores compared to U-Net and Tiramisu are really small and shows variation in the cross-validation experiments. I understand that large structures, small Dice improvements can indicate large local differences. However, 0.0003 improvement is rather small. This also questions the examples shown in Figure 3. How do the errors of the SegCaps look like? Given these quite modest — assuming they are not just due to chance — improvements, I think it is difficult to conclude that the proposed algorithm “fundamentally improves” the current state-of-the-art. 

Overall, I think it is nice idea to extend capsules to image segmentation. Article is well written and the method is clearly explained. Evaluation is the main weakness and does not support the conclusions authors would like to reach in my opinion. Despite the problems, it would be very nice to see this article in the conference and to discuss with the authors. 
review_title: Nice extension of capsules for segmentation but not very convincing evaluation

final_score: rating: 4: Top 50% of accepted papers, clear accept
review_content: Overall, the contribution as presented is more relevant to the advancement of neural network architectures and methods than it is to the field of medical image segmentation. 

* Pro: The approach is thoughtful, novel, and will probably lead to subsequent papers that will qualify the proposed methods on more data and more difficult problems (like multi-class/multi-label segmentation, 3D volumetric approaches).
* Pro: The paper is well-written and exhaustive in the description of its implementation, and as soon as the code is released (which has been promised), it will help other researchers transfer the approach to own data.

* Con: In particular for the U-Net, there are various ways to reduce parameters, the simplest being to use fewer layers. What would performance have looked like then? Isn't the interesting question what is gained by the novel architecture against comparably large (in terms of parameters) other architectures? 
* Con: The mere average dice score over hundreds of cases doesn't tell you much if it only improves by so little. It would be more interesting to see the distribution over cases or some other presentation to better understand the systematics of improvement. From the numbers in Table 1 and comparison in Fig. 3 I would suspect that in other cases, there are counter examples where the U-Net is "better".

* Remark: The arrows in Fig. 2 "shortcut connections" are differently coloured in legend and graphics.
review_title: Some novelty, some things to be desired

Decision: Oral