Review for 10.1186/s12876-023-03067-w (2024)

Title: A newly developed deep learning-based system for automatic detection and classification
of small bowel lesions during double-balloon enteroscopy examination
Version: 4 Date: 29 Aug 2023
Reviewer's report:
Dear authors,
Thank you for your point-to-point reply and critical review of your article.
I agree that it has approved much, however I still have some comments. The language is much
approved, however, some medical terms and explanation seem confusing, as if it the language
assessment was not performed by someone with a medical background.
Study design:
Add to section 1.1 that the performance of the models was tested by evaluation of 65 videos,
and compared to 8 human observers of which 4 novices and 4 experts.
Image preprocessing:
A gold standard is an international accepted 'best' evaluation, it is not something one can decide
on locally. You could describe that the three experts reached a consensus on  classification of
the images.
The method section is become a bit messy,  I would suggest one part focusing on the development
of the CNN including assessment of the images by the 3 experts and how the cut off value
works for the ResNet. Another part on the video test set, and the human readers which evaluated
the videos. I expect that the 4 experts in the video test were different from the 3 experts that
developed the classification, which I would put emphasis on in the text.
Results:
The first part of the results is actually a description of the methods, move or remove.
Supplementary table 9 and figure 4: consider moving to the results instead of supplements; since
it is enlightening for the main topic of your article
Generally I would prefer presenting the results in tables, over a long text with many percentages
in the result section.
Page 15 line 2/3: explanation of the results, belongs in the discussion section.
Table 3: the number of videos is confusing as well as the statistics
In supplementary table 5 the different pathologies are distributed as 2 x diverticulas, 30x
protruding lesions, 25x erosions&ulcera, 8x angioectasia. How can the percentages than be as
they are described in table 3? Either supplementary table 5 or table 3 is incorrect.  Can you give
an explanation or correct the tables?
Discussion:
I would like an explanation why you think experts are worse in detecting angioectasia than other
lesions.
Another possible point of discussion is that only experts from RHWU were used.
Underscore that the model was strongly dependent on the classification of lesions by the experts,
which of course are human.
Minor comments:
Use the same term throughout the entire paper,  since the main focus is the CAD system I think
it confusing if you change it with CNN and than back to CAD system. I would propose that you
describe in your methods and maybe discussion that it is a CNN; but use the term CAD system
for all other instances.
The same counts for changing present tense and past tense, for example in the Method section:
Model 1 training set includes, next sentence Model 1 image set included, and then the next
sentence model 2 includes. Choose whether you want to write in present or past tense.
Page 5 line 1: allows for tissue sampling
Page 5 line 4: apparently? (suggests that you have evidence for this, either reference to
something, or remove the word)
Page 5 line 8/9: are not likely to be detected during the same procedure. This will delay the
detection and treatment of bleeding lesions, which increases the risk of recurrent bleeding and
repeat examinations.
Page 5 line 14: have been used (delete well)
Page 5 line 19: CAD will significantly improve (reference for this argumentative sentence?)
Page 6 line 4-6: small bowel lesions in both DBE images and videos. This system might increase
the diagnostic yield for small bowel lesions that are responsible for gastrointestinal bleeding,
and help reduce the burden of repeat DBE.
Page 7 line 9: will output? (output is not a verb)
Page 8 line 12/13 remove, add this information to the study design section
Page 15 line 18: delete innovatively
Page 15 line 22: The CAD system performed similar to experts' evaluations, and better than
novices. It might be a useful tool, to improve diagnostic yield in DBE, especially for novices.
Page 16 line 3: delete handcraft oversight, use 'missed lesions'
Page 16 line 5: delete "responsible for bleeding "
Page 16 line 6: delete timely, use 'real-time'
Page 16 line 13: "previous studies", which studies are you referring to? Use references.
Page 16 line 15: "satisfactory results were achieved", according to who? what are you referring
to?
Page 16 line 16: delete kinds, use classes
Page 16 line 17: delete " but it still performed better than the average level of endoscopists
Page 17 line 7: explain better why it might not be consistent with that in the real world
Page 17 line 14: "one type of lesions" or "a single type of lesions"
Page 17 line 15: delete multi, use multiple
Page 17 line 17: delete detection precision, use accuracy
Page 17 line 18: "To improve our system - training sets." How is this sentence connected with
what you just wrote, it feels as if some information is missing in the text.
Page 18 line 18: Delete Good results had been achieved with a video test set.
Page 18 line 19: in clinical? - in daily clinical routines? or in a clinical setting?
Page 20 line 10: How often did you encounter multiple lesions in 1  image?
Page 20 line 15: studies instead of study
Page 20 line 22-page 21 line 2: we developed a novel computer-aided diagnostic system for
small bowel lesions in DBE. This CAD system can assist especially novice endoscopists in
increasing their diagnostic yield.
Supplementary tables: consider combining table 1 and 2, as well as 3 and 4.