Review for 10.1186/s12876-021-01883-6 (2021)

Reviewer’s report
Title: 
Systematic Review and Meta-analysis:Real-world Data Rates of Deep
Remission With Anti-TNFα in Inflammatory Bowel Disease
Version: 
0 
Date: 
15 Apr 2021
Reviewer's report:
PEER REVIEWER ASSESSMENTS: 
OBJECTIVE - Full research articles: is there a clear objective that addresses a
testable research question(s) (brief or other article types: is there a clear
objective)?
Yes - there is a clear objective
DESIGN - Is the current approach (including controls and analysis protocols)
appropriate for the objective?
No - there are minor issues
EXECUTION - Are the experiments and analyses performed with technical rigor
to allow confidence in the results?
No - there are minor issues
STATISTICS - Is the use of statistics in the manuscript appropriate?
Yes - appropriate statistical analyses have been used in the study
INTERPRETATION - Is the current interpretation/discussion of the results
reasonable and not overstated?
Yes - the author's interpretation is reasonable
OVERALL MANUSCRIPT POTENTIAL - Is the current version of this work
technically sound? If not, can revisions be made to make the work technically
sound?
Probably - with minor revisions
PEER REVIEWER COMMENTS: 
GENERAL COMMENTS: 
* What is your overall impression of the study?
Interesting subject and detailed assessments and analyses were presented.
* What have the authors done well?
A large number of papers have been screened and reviewed. The meta-analysis
should provide helpful information on this biologic therapy for inflammatory
bowel disease for clinicians and patients.
* In what ways does it not meet best practice?
Minor: I am not sure that 'deep remission' is fully understood as 
clinical term. 
It
should be characterised as including 
clinical remission, biomarker remission and
mucosal healing.
Major: Not current evidence - the search date is July 2019, updated searches
should be carried out and the findings integrated in to the review.
The inclusion of non RCT trials is likely to lead to several biases including
selection and reporting bias. 
I have made further comments on this.
The impact of bias and confounders on the results has not been fully explored in
the text. 
There is no overall assessment of the quality of the body of evidence or certainty
in the findings.
REQUESTED REVISIONS: 
Additional revisions
METHODS
It is unclear from the Methods section whether studies with a control group
(nonrandomised controlled group studies) were explicitly sought, and how such
studies were dealt with in the meta-analysis. 
It is unclear if the authors made any effort to search or obtain missing data, such
as missing outcome data, or made any data conversions prior to meta-analysis. 
Did the authors search grey literature and nursing literature? The choice of
databases for electronic searching is limited and does not include foreign
language databases or nursing or grey literature databases. 
Potentially many
studies could be missed by adopting such a limited search strategy. 
The full
search strategy was not supplied with my copy of the paper however the search
terms indicated would not provide an exhaustive search of the literature. Also,
the 'deep remission' can include biomarkers. If this term was excluded then
studies that included biomarkers as an indicator of remission are likely to have
been missed and this is a limitation.
Did the authors include any specific search terms for adverse events?
The search date is July 2019, meaning that this is not completely up-to-date
evidence. Updated searches should be carried out and the findings integrated in
to the review.
RESULTS
TABLE 1. 
This could be improved by annotating in this table which study
characteristics were grouped together for the analyses. Table 1 could indicate if
any studies employed a control group. The decision to describe studies as
retrospective and prospective (as opposed to describing the studies by labels as
e.g. 'cohort' ) in table 1 is good. 
The PRISMA flowchart is helpful, however no list of references for the excluded
studies and reasons for exclusion is provided. I accept that for 741 studies this
would be a very long table, however some indication of which studies were
excluded, with reasons, should be provided, perhaps in a supplementary data
table.
Bias and confounders: it is good to see that the authors assessed study bias using
the Newcastle Ottawa scale. 
For the Newcastle Ottawa scale I prefer the use of
points rather than "stars". The table of findings for the Newcastle Ottawa
assessment of risk bias (supplementary Table 2) is useful, but the reporting of
NOS assessed bias in the results section is inadequate; there needs to be an
indication of which studies were at risk of bias, what impact this might be and
any further analysis that were conducted in order to investigate or mitigate
impact of study bias. At present, it is unclear if additional analyses were planned
or carried out to assess the impact of study bias. 
Heterogeneity: the authors have applied a random effects model to their analyses
and assess between study heterogeneity. 
However, the results of these
assessments and investigations into possible causes of heterogeneity have not
been provided in the review. 
While reasons for heterogeneity are discussed in the
discussion, the impact of any statistical heterogeneity encountered in the meta-
analysis on the overall results and the implications for patients and clinicians has
not been discussed fully enough. This should be amended before publication.
There is no description of any methods used to assess certainty (or confidence)
in the body of evidence for each outcome. 
The preferred method would be
GRADE, and for non-RCT evidence the methodological quality of the included
is likely to be low (or even very low), therefore the confidence in the certainty of
the findings is that further research is very likely to have an important impact,
which is likely to change the estimate of effect. Ideally the authors would
provide some statements, in the Discussion or Results section for each separate
analysis, about the confidence in the certainty of the findings. Without these
assessments, the usefulness of the review is limited.
The results section is otherwise well written. 
Although effect sizes are given, it is
unclear whether the size of the effect estimate represents a promising result for
the intervention and issues such as wide confidence intervals or small studies can
make it difficult to draw conclusions. I advise that the authors refer to section
15.6.4 of the Cochrane Handbook. 
This provides useful statements for phrasing
conclusions based on effect size and certainty of the evidence.
There is no mention of potential adverse effects of anti TNF- 
α.
 One of the
potential advantages of using nonrandomised controlled trial data, could be that
adverse events are more fully reported. 
This omission detracts from the
usefulness of the review for clinicians and patients.
DISCUSSION
The potential influence of study bias on the results is not adequately addressed.
There needs to be a fuller discussion of how potential confounders were
addressed and how clinical and statistical heterogeneity between studies and
other potential biases in the studies may have influenced the results. 
This
omission should be addressed, throughout the review (Results and Discussion
sections) before publication.
There is only a limited discussion of the implications of the results for policy
practice and future research. This section of the discussion could be structured
using the 'EPICOT' tool, or the Cochrane Handbook chapters on interpretation of
results provide helpful information about this, which can be applied to non-
Cochrane reviews such as this one.
As part of my peer-reviewed comments I have provided comments against the
PRISMA 2020 checklist and the PRISMA 2020 abstract checklist which I have
forwarded to the editorial office. 
I strongly advise that the review authors use the
PRISMA checklist when making modifications to their review before
publication, and perhaps provided the checklists as supplementary documents for
inclusion online (subject to the agreement of the journal's editor).
ADDITIONAL REQUESTS/SUGGESTIONS: 
The design and decision to use non-randomised controlled trial data is
acceptable, and the systematic review and analyses appear to have been
conducted appropriately.
However the rationale for not including RCTs is unclear. 
While the authors, in
the discussion accept that the non-inclusion of RCTs is a limitation, there is
insufficient discussion of the potential biases in the conduct and reporting of the
review because of bias and confounders in the included studies. Principally these
biases will be selection bias and reporting bias. 
On the whole this is an interesting piece of work and with some editing, could be
a useful addition to the knowledge base on biologics for inflammatory bowel
disease.
Publons Reviewer Recognition
. Springer Nature can send verification of this
review directly to Publons (a subsidiary of Clarivate Analytics). If you would
like to take advantage of this service, please click on the “Yes” option below.
Your name, email address, title of the reviewed manuscript, name of the journal,
and date of your review submission (the “Review Data”) will then be transmitted
to Publons after the final decision on the manuscript has been made. If you have
already registered at Publons, they will notify you of the receipt of this review
and update your profile 
as per your settings and their policy. If you are not
registered with Publons, you will receive an email from them asking you to
register in order for them to be able to recognize your review on your new
profile page. Publons may use the Review Data to generate derivative metadata
for the benefit of Publons and you as a reviewer, carefully considering the
sensitivity of such information. For example, Publons may verify your record as
a reviewer by updating your profile published on its webservice if you have
registered for such service or help editors to identify candidate reviewers. Please
find the details of processing in Publons’ privacy policy
https://publons.com/about/terms
No
Declaration of competing interests
Please complete a declaration of competing
interests, considering the following questions:
1
. 
Have you in the past five years received reimbursements, fees, funding, or
salary from an organisation that may in any way gain or lose financially
from the publication of this manuscript, either now or in the future?
2
. 
Do you hold any stocks or shares in an organisation that may in any way
gain or lose financially from the publication of this manuscript, either now
or in the future?
3
. 
Do you hold or are you currently applying for any patents relating to the
content of the manuscript?
4
. 
Have you received reimbursements, fees, funding, or salary from an
organization that holds or has applied for patents relating to the content of
the manuscript?
5
. 
Do you have any other financial competing interests?
6
. 
Do you have any non-financial competing interests in relation to this
paper?
If you can answer no to all of the above, write 'I declare that I have no competing
interests' below. If your reply is yes to any, please give details below.
This reviewer has been recruited by a partner organization, Research Square.
Reviewers with declared or apparent competing interests are not utilized for
these reviews. This reviewer has agreed to publication of their comments online
under a Creative Commons Attribution License attributed to Research Square
and was paid a small honorarium for completing the review within a specified
timeframe. Honoraria for reviews such as this are paid regardless of the reviewer
recommendation.
Reviewer Publication Consent.
 I agree for my report to be made available
under an Open Access Creative Commons CC-BY License (
http://creativecommons.org/licenses/by/4.0
) if this manuscript is accepted for
publication. Any comments that I do not wish to be included in the published
report have been included as confidential comments to the editor, which will not
be published.
If you are not happy for us to publish this report, please contact the
editorial office before completing the review.
If you wish, you can include your
name in your published report. Please note you must decide whether to include
your name at the start of the process and confirm this decision whenever
prompted. If you change your decision later, we will not be able to publish your
name.
I agree to the terms of the CC-BY 4.0 license; please 
do not
 publish my name
with my report. (default)
Is the study design appropriate to answer the research question (including the use
of appropriate controls), and are the conclusions supported by the evidence
presented?
No
Are the methods sufficiently described to allow the study to be repeated?
No
Is the use of statistics and treatment of uncertainties appropriate?
Yes
Is the presentation of the work clear?
No
Are the images in this manuscript (including electrophoretic gels and blots) free
from apparent manipulation?
Yes