Venue: MIDL 2018 Conference

Reviews:

final_score: rating: 4: Top 50% of accepted papers, clear accept
review_content: Authors present their work on identifying epileptic brain regions in MR-negative patients. The proposed unsupervised outlier detection approach is a common strategy in this field and seems promising and suitable for this task. In this work, authors have replaced the hand-crafted features with automatically learned features. The paper is clear and well-written.

Abstract: some quantitative results should be included in the abstract. Currently, the abstract is a summary of introduction/methods, but does not contain any results nor discussion/conclusion.

Introduction: authors make a valid point that currently used features are insufficient to detect all epilepsy lesions, resulting in MR-negative cases that do have epilepsy but no lesion can be found. Some description of the currently use (hand-crafted) features could be included in the introduction, to give the reader some background information on those features. These features (e.g. by Huppertz, who is not cited) are used later in the results and table 1.

Methods: authors state that their method has a high number of classes: each individual voxel in the brain. However, these are not all independent from each other, as neighbouring voxels share features/information. Is this dependence between neighbouring voxels taken into account in this method? Are the trained oc-SVM classifiers of neighbouring voxels similar to each other?

Experiments: for evaluation, it seems that authors consider an MR-negative lesions as "detected" when their method produces a detection on the pre-surgery image in the resected zone. The size of the resected zone can vary between patients and can also be quite large. How specific are the detections and how does it compare to the size of the resected zone? Is there a clear detection in the centre of the resected zone or somewhere at the border?
review_title: Solid conference paper

final_score: rating: 3: Marginally above acceptance threshold
review_content: Summary
This paper tackles the problem of detection of subtle lesions in MRI formulated as an outlier detection approach applied to the problem of epilepsy lesion detection.
The proposed approach tries to overcome the need of manual annotations for supervised approaches by only using non-pathological cases for training, which allows unsupervised learning.
The proposed method is an improvement of previous approach, based on SVM classifier, but now hand-crafted features are replaced with a representation learned with neural networks.
Siamese network is modified by removing the label "similar/dissimilar", because all patches are "similar" in the designed approach.
Therefore, the loss function is modified to include the cosine similarity of encoded representation and the mean squared error between input and output patches.

Strong points
* Exploiting data in an unsupervised fashion.
* Authors claim this is the first time this approach is applied to multiparametric MRI data.
* The modified loss function is interesting, and allows to avoid learning wrong representation that fullfill the loss in a trivial way (all zeros, as mentioned in the paper).

Weak points
* Underlying idea is not novel as is based on previous work.
* What is the main difference between the proposed loss function and other loss functions used in autoencoders? Isn't the different between input and reconstructed patch considered already?
* A threshold of 82 voxels is used, but not justified. Why?
* Same for the 15 clusters considered. Why 15?
* How sensitive is this method to the registration part? How can a voxel-wise correspondence be ensured? It seems that this method stronglty relies on the output of SPM. Could other pipeline be used, and how is the method robust to the use of other systems (in case for example SPM is not available or not allowed, for example in a clinical setting)?
* Image rescaling between 0 and 1 is done at image level, or statistics on the training set are used?
* In Figure 4, case B and C have clusters in positions that are close to the lesion, but not containing lesion voxels, apparently.
* The performance of SVM with hand-crafted feature is reported to be 70% in [11]. Does this mean that the proposed approach under-perform the previous one? Are the datasets used in both papers the same?

Extra comments
* Having an embedded represteation of 2x2, still keeps spatial information. Other papers have a 1x1xN feature map in this position of the encoder. What would be the difference?
* Could a semi-supervised approach be used here as well? It seems to work well when hand-crafted fetaures are used, so it could boost the eprformance of this approach, if some supervision is introduced.
review_title: Regularized siamese neural network for unsupervised outlier detection on brain multiparametric magnetic resonance imaging: application to epilepsy lesion screening

final_score: rating: 4: Top 50% of accepted papers, clear accept
review_content: This work proposed a framework for epilepsy lesion screening, posed as an outlier detection problem. This work builds on the current state of the art, which involves the handcrafting of features for specific pathologies; instead, the authors exploit the representations learned by neural networks to generalize this task. The methods are sound and the results are promising, but a few clarifications would be beneficial to the paper.

Pros:
- Sound method
- Definite improvement on the state of the art
- Novel approach to dealing with unbalanced datasets with limited labelled data
- well thought-out loss function

Cons:
- Could use more of a literature review
- Grammar revisions required

Some other comments:
- "In this case the number of classes would be equal to the number of voxels in a brain volume (around 4 millions) but the number of samples per class would be equal to the number of subjects." and ". Each voxel is associated with a classifier, hence the number of classifiers is equal to the number of voxels in a volume (around 4 million voxels)."
As the voxels in an image volume are spatially correlated with their neighbours, is there any way to exploit this property and reduce the parameters and number of classifiers required?
- Figure 4: Not sure what caption labels refer to
- How was the post-processing size threshold selected? (82) is this common in similar literature, or determined experimentally? Does this affect results at all?
- Why keep 15 clusters in post-processing? 
- The "detection" metric could use some elaboration. It appears as though it's a common metric for the literature, but it is unclear for a deep learning crowd. Does it mean that it detected that a subject had a lesion? Or overlap with the ground truth lesions? Unclear.
- Figure 4: Can subjects have more than one lesion? This should be clarified. If yes, other examples with greater than one lesion should be included.

Good work, but a more thorough literature review would definitely be beneficial.
review_title: Good work, just needs a few clarifications

Decision: Oral