Venue: ICLR 2018 conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: The authors compare a standard DL machine (AlexNet) with a custom CNN-based solution in the well known tasks of classifying lung tumours into benign or cancerous in the Luna CT scan dataset, concluding that the proposed novel solution performs better.
The paper is interesting, but it has a number of issues that prevents it from being accepted for the ICLR conference.

First, the scope of the paper, in its present form, is very limited: the idea of comparing the novel solution just with AlexNet is not adding much to the present landscape of methods to tackle this problem.
Moreover, although the task is very well known and in the last few year gave rise to a steady flow of solutions and was also the topic of a famous Kaggle competition, no discussion about that can be found in the manuscript.
The novel solution is very briefly sketched, and some of the tricks in its architecture are not properly justified: moreover, the performance improvement w.r.t . to AlexNet is hardly supporting the claim.
Experimental setup consists of just a single training/test split, thus no confidence intervals on the results can be defined to show the stability of the solution.
The whole sections 2.3 and 2.4 include only standard material unnecessary to mention given the target venue, and the references are limited and incomplete.
This given, I rate this manuscript as not suitable for ICLR 2018.
review_title: Paper with interesting ideas, but far from meeting ICLR level.

final_score: rating: 3: Clear rejection
review_content: The paper compares AlexNet and a custom CNN in predicting malignant lung nodules, and shows that the proposed CNN achieves significantly lower false positives and false negative rates.

Major comments

- I did not fully understand the motivation of the custom CNN over AlexNet. 

- Some more description of the dataset will be helpful. Do the 888 scans belong to different patients, or same patient can be scanned at different times? What is the dimensionality of each CT scan?

- Are the authors predicting the location of the malignant nodule, or are they classifying if the image has a malignant nodule? How do the authors compute a true positive? What threshold is used?

- What is 'Luna subsets'? What is 'unsmoothed and smoothed image'?

Minor comments

- The paper is difficult to read, and contains a lot of spelling and grammatical errors.
review_title: difficult to read, need more details

final_score: rating: 2: Strong rejection
review_content: This paper compares 2 CNN architectures (Alexnet and a VGG variant) for the task of classifying images of lung cancer from CT scans. The comparison is trivial and does not go in depth to explain why one architecture works better than the other. Also, no effort is made to explain the data beyond some superficial description. No example of input data is given (what does an actual input look like). The authors mention "the RCNN object detector" in step 18, that presumably does post-processing after the CNN. But there is no explanation of that module anywhere. Instead the authors spend most of the paper listing in wordy details the architecture of their VGG variant. Also, a full page is devoted to detailed explanation of what precision-recall and Matthews Correlation Coefficient is! Overall, the paper does not provide any insight beyond: i tried this, i tried that and this works better than that; a strong reject.
review_title: a trivial comparison of 2 CNN models for lung cancer detection on CT scans

Meta-review: Pros:
- Addresses an important medical imaging application
- Uses an open dataset

Con:
- Authors do not cite original article describing challenge from which they use their data: https://arxiv.org/pdf/1612.08012.pdf , or the website for the corresponding challenge: https://luna16.grand-challenge.org/results/
- Authors either 1) do not follow the evaluation protocol set forth by the challenge, making it impossible to compare to other methods published on this dataset, or 2) incorrectly describe their use of that public dataset.
- Compares only to AlexNet architecture, and not to any of the other multiple methods published on this dataset (see: https://arxiv.org/pdf/1612.08012.pdf).
- Too much space is spent explaining well-understood evaluation functions.
- As reviewers point out, no motivation for new architecture is given.

Decision: Reject