Review for 10.1186/s12876-021-02082-z (2022)

Reviewer’s report
Title: 
A case for improved assessment of gut permeability – a meta-analysis
quantifying the lactulose:mannitol ratio in coeliac and Crohn’s disease
Version: 
0 
Date: 
15 Oct 2020
Reviewer's report:
Gan et al provide an informative systematic review and meta-analysis of studies
quantifying the lactulose:mannitol ratio (LMR) in coeliac and Crohn's diseases.
The authors also compare this ratio in untreated and treated coeliac and active
and inactive Crohn's disease to healthy controls. The authors suggest that this
ratio has a role in diagnostic testing, particularly in coeliac disease. However, I
have concerns about the methodology used by the authors, accuracy of the data
extraction, and reporting of the work that should be addressed prior to
publication.
Major Comments:
Methods:
* Study Quality:
o A table describing the risk of bias for each domain within each study should be
included. Please see the example included in the description of Item 19 in the
PRISMA Statement Explanation and Elaboration document
(https://journals.plos.org/plosmedicine/article?
id=10.1371/journal.pmed.1000100). Reporting overall risk of bias for individual
studies does not allow the readers to assess specific threats to the validity of each
study.
o The risk of bias for studies evaluating the LMR as a diagnostic accuracy tool
in coeliac disease should be assessed using QUADAS-2
(https://www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-
2/).
* Definition of Active and Inactive Crohn's Disease:
o The authors specifically state using differentiating between active and inactive
Crohn's disease using a score of 150 on the Crohn's Disease Activity Index
(CDAI). However, I have several concerns with this statement:
 Table S1 only mentions that the CDAI was used in two studies (Marcilio et al
1998 and Siagalet et al 2013). The only other study listing a disease activity
score was Zamora et al 1999. This study used the Pediatric Crohn's Disease
Activity Index (PCDAI), although the authors list this as a modified Harvey
Bradshaw Index. It is unclear how other studies differentiated between active and
inactive disease.
 In a spot check of selected studies, the threshold of 150 on the CDAI is
incorrectly used to differentiate between active and inactive disease:
* The authors state that Haas et al 2009 compares patients with inactive Crohn's
disease to controls. However, 8 of the 29 Crohn's disease patients included in the
study were on steroids and 8 were acutely ill at the time of the study. While the
mean CDAI among included patients was 127 (less than the threshold of 150),
the maximum CDAI among included patients was >300 (a score indicative of
moderate-to-severe Crohn's disease). This, therefore, is not a true comparison of
inactive Crohn's disease and controls.
* The Wild et al 2003 study is a comparison of patients in an acute disease flare
before and after receiving a course of steroids - the authors state that this is a
comparison of active and inactive Crohn's disease. However, only 22 of 30
patients included in the study reached steroid-induced clinical remission. This
means that 8 patients continued to have active disease and the comparison
presented by the authors is not truly a comparison of active and inactive Crohn's
disease.
* Given the differences in the methods used to estimate the LMR across studies,
the authors should report the pooled standardized mean difference, rather than
the weighted mean difference. This estimate should be less vulnerable to
differences across assays.
* In order to calculate a sensitivity and specificity for the LMR in diagnosing
coeliac disease, a threshold would be required, whereby an LMR above this
threshold would be considered to correspond to a coeliac diagnosis while an
LMR below this threshold would not be considered coeliac. Given the
heterogeneity in values and methodology observed across the included studies, it
is likely that these thresholds varied across studies. Additional information about
these thresholds is required. This corresponds to item 14 on the PRISMA-DTA
checklist (extension of PRISMA for diagnostic accuracy tests:
http://www.prisma-
statement.org/documents/PRISMA%20DTA%20Checklist.pdf)
Data Accuracy:
* In a spot check of some studies, errors were noted in (see below for more
detailed comments):
o The LMR values
o The method of diagnosis
* All data should be verified for accuracy prior to publication.
Discussion:
* The authors state that "Differences in LMR were more significant when using
the 2:1 ratio than the 5:2 ratio." While the magnitude of the difference in the
LMR for untreated coeliac and healthy controls was greater for the 2:1 ratio and
the p-value was slightly smaller, the effect estimates and confidence intervals for
these estimates are quite similar. Further, there was high heterogeneity in most of
these comparisons and no statistical test was conducted that would allow authors
to reach this conclusion. The authors should be cautious about over-interpreting
their findings.
* Can the authors comment on how intestinal permeability may vary across
disease characteristics? For example, how would intestinal permeability vary in a
Crohn's patient with long-standing mucosal healing relative to a patient who had
permeability measured following a course of steroids to induce remission? These
two patients would be treated identically as patients with 'inactive' disease based
on the analyses conducted in this systematic review.
Minor Comments:
* The number of decimal places should be consistent throughout the manuscript,
including in the text, tables, and forest plots.
Abstract:
* The authors provide the pooled LMR values for each disease state (healthy
controls, treated and untreated coeliac disease, and active and inactive Crohn's
disease) while only stating that the differences between groups were significant.
If space allows, readers would benefit from seeing the values for the
comparisons between groups.
Methods:
* The full literature search should be provided, including the specific
combination of MeSH terms and keywords used. Please see the example
included in the description of Item 8 in the PRISMA Statement Explanation and
Elaboration document (https://journals.plos.org/plosmedicine/article?
id=10.1371/journal.pmed.1000100).
* The authors state that secondary outcomes were based on "different
concentrations of lactulose and mannitol in the aforementioned groups." Stating
that all possible concentrations of lactulose and mannitol were permitted in the
primary analysis would improve the clarity of the primary outcome.
* Did all included studies report means and standard deviations? If some
reported medians with either interquartile ranges or ranges, how were these
handled? How were paired data handled in the analysis (e.g., for Wild et al 2003,
where LMR measurements were repeated for each individual before and after a
course of steroids)?
* Can the authors clarify the 'weighted mean' and 'standard mean' LMR values
for each group? Only a single pooled mean LMR is reported for each group.
* The authors report weighted mean differences for within-study comparisons.
Weighted mean differences typically refer to the pooled estimate, with the
'weighted' component referring to the different weights applied to each study in
the overall estimate. The authors should correct this terminology.
Results:
* The authors state that 40 studies were included: 21 Crohn's disease and 19
coeliac. However, two studies report data for both Crohn's disease and coeliac
disease. The total number of studies included should be clarified.
* It would be helpful to add a description of the interventions study participants
were subjected to in Tables S1 and S2 for the RCTs and observational
interventional studies (e.g., Wild et al 2003 where LMR was measured before
and after a course of steroids in patients with Crohn's disease) included in this
review. 
* The direction of the differences in the LMR is unclear when they are described
in the text. For example, in the sentence "The change in LMR between healthy
controls and treated coeliac disease (5 studies) was 0.009 (95% CI 0.004-
0.14…)" it is not clear if the LMR was higher in the healthy controls for those
with coeliac disease. It is also not clear if these estimates are weighted or
standardized mean differences. In addition, the confidence intervals are not clear
as written. For example, in the first paragraph of the section '5:2 solute used in
Crohn's disease and coeliac disease', it is not clear if the confidence interval
should be '-0.083 to -0.331' or '-0.083 to 0.331'. 
* Several studies in Table 2 list multiple comparison groups (first-degree
relatives, non-blood relatives, and spouses of patients). Were these non-Crohn's
disease controls included in the healthy controls? Were they excluded? The
analogous table in coeliac disease does not list these same comparisons (e.g.,
Vogelsang et al 2001 also described the LMR in patients' spouses).
* Vilela et al 2008 is included in Figure 1 twice.
* In Table S1, the column 'Method of diagnosis' should be split into two
columns, with one column describing how a diagnosis of Crohn's disease was
confirmed and the other column describing how studies defined active vs.
inactive Crohn's disease. The Crohn's Disease Activity Index is not a tool that
can be used to diagnose Crohn's disease.
* In Table S2, the column 'Method of diagnosis' should be clarified. Specifically,
the Crohn's Disease Activity Index cannot be used to diagnose coeliac disease.
Further, neither clinical symptoms, radiology, or endoscopy are sufficient for a
coeliac diagnosis. A diagnosis of coeliac disease should be biopsy-proven or
based on serology (both anti-TTG and anti-EMA). In a spot check of Vogelsang
et al 2001, the diagnosis of coeliac disease was based on histology. However,
Table S2 describes this study as basing coeliac diagnosis on non-specific clinical
symptoms.
* Tables S1 and S2 should include additional details about how participants were
recruited to the study. For example, were cases hospitalized patients, recruited
from tertiary care centres, or from another source? Similarly, how were controls
identified? In Vogelsang et al 2001, controls were individuals with non-specific
gastrointestinal symptoms without a formal diagnosis. This control group may
differ from a control group without any gastrointestinal conditions.
* Tables 1 and 2 should include measures of within study variability (standard
deviations, standard errors, interquartile ranges, or ranges) for the reported
LMRs. These tables should also report the number of participants used to
calculate the mean or median LMR for each group within each study.
* In a spot check of the Wild et al 2003 study, there are errors in the reported
LMRs. For inactive Crohn's, the LMR should be 0.046 instead of 0.026. This
study also reported data on healthy controls and was missed by the authors.
* A footnote should be added to Tables 1 and 2 describing the L:M solute ratio
used in each study.
Discussion:
* Can the authors comment on the potential role of LMR in differentiating
between pathologies in a patient newly presenting with gastrointestinal
symptoms?
Other Comments:
* Some of the studies included in the meta-analysis are missing from the
reference list, including Marcilio et al 1998 and Vilela et al 2009.
Are you able to assess any statistics in the manuscript or would you
recommend an additional statistical review?
If an additional statistical review
is recommended, please specify what aspects require further assessment in your
comments to the editors.
I am able to assess the statistics
Declaration of competing interests
Please complete a declaration of competing
interests, considering the following questions:
1
. 
Have you in the past five years received reimbursements, fees, funding, or
salary from an organisation that may in any way gain or lose financially
from the publication of this manuscript, either now or in the future?
2
. 
Do you hold any stocks or shares in an organisation that may in any way
gain or lose financially from the publication of this manuscript, either now
or in the future?
3
. 
Do you hold or are you currently applying for any patents relating to the
content of the manuscript?
4
. 
Have you received reimbursements, fees, funding, or salary from an
organization that holds or has applied for patents relating to the content of
the manuscript?
5
. 
Do you have any other financial competing interests?
6
. 
Do you have any non-financial competing interests in relation to this
paper?
If you can answer no to all of the above, write 'I declare that I have no competing
interests' below. If your reply is yes to any, please give details below.
I declare that I have no competing interests.
Statement on potential review bias
 Please complete a 
statement on potential
review bias, considering the following questions:
1. Did you co-author any publication with an author of this manuscript
in the last 5 years? 2. Are you currently or recently affiliated at the
same institution as an author of this manuscript?
If you can answer no to all of the above, write 'I declare that I did not publish
with these authors in the last 5 years and also meet the affiliation criteria”. If
your reply is yes to any, please give details below.
I declare that I did not publish with these authors in the last 5 years and also
meet the affilliation crtieria.
Reviewer Publication Consent.
 I agree for my report to be made available
under an Open Access Creative Commons CC-BY License (
http://creativecommons.org/licenses/by/4.0
) if this manuscript is accepted for
publication. Any comments that I do not wish to be included in the published
report have been included as confidential comments to the editor, which will not
be published.
If you are not happy for us to publish this report, please contact the
editorial office before completing the review.
If you wish, you can include your
name in your published report. Please note you must decide whether to include
your name at the start of the process and confirm this decision whenever
prompted. If you change your decision later, we will not be able to publish your
name.
I agree to the terms of the CC-BY 4.0 license; please 
do not
 publish my name
with my report. (default)
Publons Reviewer Recognition
. Springer Nature can send verification of this
review directly to Publons (a subsidiary of Clarivate Analytics). If you would
like to take advantage of this service, please click on the “Yes” option below.
Your name, email address, title of the reviewed manuscript, name of the journal,
and date of your review submission (the “Review Data”) will then be transmitted
to Publons upon publication of the manuscript. If you have already registered at
Publons, they will notify you of the receipt of this review and update your profile
as per your settings and their policy. If you are not registered with Publons, you
will receive an email from them asking you to register in order for them to be
able to recognize your review on your new profile page. Publons may use the
Review Data to generate derivative metadata for the benefit of Publons and you
as a reviewer, carefully considering the sensitivity of such information. For
example, Publons may verify your record as a reviewer by updating your profile
published on its webservice if you have registered for such service or help
editors to identify candidate reviewers. Please find the details of processing in
Publons’ privacy policy 
https://publons.com/about/terms
No
Is the study design appropriate to answer the research question (including the use
of appropriate controls), and are the conclusions supported by the evidence
presented?
1
Are the methods sufficiently described to allow the study to be repeated?
2
Is the use of statistics and treatment of uncertainties appropriate?
2
Is the presentation of the work clear?
2
Are the images in this manuscript (including electrophoretic gels and blots) free
from apparent manipulation?
1