Venue: MIDL 2018 Conference

Reviews:

final_score: rating: 5: Top 15% of accepted papers, strong accept
review_content: This paper introduces a new approach for the segmentation of the femoral bone (FB), tibial bone (TB), and the respective femoral and tibial cartilage (FC,TC) from MRIs.  The technique proposed for such task is based on a combination of statistical shape model (SSM) and deep learning, involving the following steps: 1) slice by slice 2-D FB and TB segmentation using a 2-D U-net (CNN) trained with a Dice loss, where the input contains 8 slices above and below the slice to be segmented (they serve as a context information for the segmentation); 2) SSM shape adjustment in 3-D that regularizes the segmentation from step (1) above; 3) sub-volume FB and TB segmentation using a 3-D U-net (CNN) trained with a Dice loss, where the results from the sub-volume segmentation are combined via majority voting in overlapping regions (i.e., volume regions that have more than one sub-volume segmented); 4) SSM-Postprocessing that regularises the FB and TB  segmentation from step (3) by removing wrongly classified foreground labels located outside the typical range of osteophytic growth; and 5) FC and TC segmentation using 3D U-net.  Experiments based on volume-based (Dice, volume difference, volume overlap error) and distance-based (average surface distance, root mean square distance, maximum distance) measures are shown on three datasets: SKI10, OAI Imorphics and OAI ZIB.  On SKI10, the proposed approach achieves the currently best results in the field.  Results on OAI Imorphics and OAI ZIB are harder to interpret since there is no comparison with any baseline, and OAI ZIB appears to be not so challenging.

I think this is a solid paper with interesting results and enough technical novelty, so I recommend it to be accepted for MIDL.  My main recommendation is for the authors to address one of the issues mentioned in Section 4: how to couple SSMs and CNNs more directly - in particular how could a joint SSM-CNN model be trained end-to-end?


review_title: Review of Automated Segmentation of Knee Bone and Cartilage using Statistical Shape and CNNs

final_score: rating: 5: Top 15% of accepted papers, strong accept
review_content: 
Summary:
A novel segmentation pipeline comprising of 2D and 3D U-Nets along with a-priori statistical shape models (SSM)  are used to segment knee bone and cartilage from MRI. The method uses SSMs for regularisation and post-processing to improve the segmentation results. The method currently ranks as the best performing method in the MICCAI SKI10 challenge, and is also evaluated on two other MRI datasets yielding competitive results.

Pros

1. Thorough literature survey is presented relating to SSM and CNN based segmentation methods. 
2. Incorporating a-priori SSMs is a novel and interesting contribution when using the two step 2D and 3D U-Net based segmentation. This can be of interest in other medical image applications where shape models are available.
3. Extensive evaluation of the method on three datasets 
4. Performance on the MICCAI SKI10 challenge is impressive. 
5. The paper is clearly written. 

Cons

1. Pointers to deep learning based methods that use some form of shape models could be useful to position the current work such as [1]. This also relates to the discussion on how to couple SSMs and CNNs in Section 4.
2. Use of SSMs obtained from SKI10 is used on OAI Imorphics in Section 3.2, which is a strong point if it is transferable but discussion on what the impact of this is missing. If SSMs are unavailable for one dataset, to what degree are SSMs from other datasets useful? 

References

[1] Oktay, Ozan, et al. "Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation." IEEE transactions on medical imaging 37.2 (2018): 384-395.
https://arxiv.org/pdf/1705.08302.pdf
review_title: Novel U-Net based segmentation pipeline that uses SSMs for regularisation and post-processing

final_score: rating: 3: Marginally above acceptance threshold
review_content: Overall:
The paper proposes a solution to the problem of knee bone and cartilage segmentation from MRI using deep learning (2D and 3D U-Nets) and Statistical Shape Knowledge (SSM). In general the idea is sound and it received state-of-the-art results in the MICCAI SKI10 challenge. My main concern is whether SSM components allow to learn CNN through back-propagation. Except that, the paper is very interesting and it is a solid contribution to the conference.

Strengths:
+ The proposed approach is interesting solution to the problem of knee bone and cartilage segmentation from MRI.
+ The method combines a deep learning approach with Statistical Shape Knowledge (SSM).
+ The results are impressive since the proposed approach obtained the best average performance in the MICCAI SKI10 grand challenge.

Remarks:
* Major
- I have big doubts whether SSM-Adjustment and a voting mask allow for gradient flow. In other words, whether these two components allow to learn CNNs through back-propagation. What is a learning procedure in the considered case?
- What is the point of presenting results for OAI datasets if there is no baseline we can compare to?

* Minor
- Why names of the last two authors are not in bold?
- The paper needs additional editing since there are missing spaces between paragraphs. 
- The authors state that a single segmentation takes ~9.5min. How slow is the application of SSM comparing to the CNN-based components? It seems that SSM is a bottleneck and this could be easily improved in the future.
review_title: A good paper

Decision: Oral