Review for 10.1186/s12876-015-0327-x (2015)

Reviewer's report
Title:Interobserver and intraobserver agreement for gastric mucosa atrophy
Version:2Date:22 February 2015
Reviewer: Richard Owen
Reviewer's report:
1. The question posed by the authors is well defined.
2. The methods described are sound, within the limitations identified later in the
manuscript. Considering the natural comparison to inter-observer agreement in
pathological assessment (as recognised by the authors), it may have been
advisable to use this background to design power statistics to confirm
observers/patients needed to achieve at least the primary study aims.
Specific issues are the use of serological testing in previously infected/current
infected groups, which may confound the groupings as defined by the authors.
Summary statistics explaining how patients were diagnosed, and with which
modality should be provided, perhaps in a supplementary format, to help readers
identify potential limitations of the patient type classification system.
The reasoning behind separating assessor groups by nationality is not
adequately explained. I accept the authors identify the 'spreading' usage of
endoscopy in Vietnam versus its establishment in Japan, but how applicable this
statement is to the presumably high-volume centres in which this study have
taken place is not clear.
3. The omission of summary statistics in the text concerns me. The data in tables
2 and 3 demonstrate overlapping SD in all measures, as would be expected in a
study of this size. This should be clearly identified in the text to avoid confusion
regarding the significance of the results.
4. No issues with figures
5. Reporting standards adequate.
6. Whilst the conclusion is suitable to the data presented, the degree of
speculation in the discussion is not well justified by the data. For example, the
data is not able to call differences between endoscopy experience levels. Also,
the comparisons between intra and inter-observer agreement are not supported.
Unfortunately, the language used becomes a little difficult to follow in para
212-221 but I think I understand the overall point that a simpler endoscopic
grading system would aid in inter-observer agreement.
7. The limitations are outlined and appropriate.
Major Compulsory Revisions
None
Minor Essential Revisions
Clarification of the type of kappa statistic used.
Addition of a summary of diagnostic methods used to group patients by H pylori
infection, ideally avoiding over-reliance on serological markers in currently
infected group.
Discretionary Revisions
It would help readers if tables included an indication of how data is being
presented (ie. mean and sd).
Some explanation of how concordance rates are used prior to the results section
would help understand how the data presented in table 1 is subsequently used to
define patient endoscopic classification.
A reference to support why the agreement levels have been set would be
appropriate, especially considering the controversy around the use of this sort of
agreement 'banding' in kappa statistic-based analysis (Landis and Koch is the
usual example).
Level of interest: An article of limited interest
Quality of written English: Needs some language corrections before being
published
Statistical review: Yes, but I do not feel adequately qualified to assess the
statistics.
Declaration of competing interests:
I declare that I have no competing interests