Reviewer's report
Title:Methodological quality of test accuracy studies included in systematic
reviews in obstetrics and gynaecology: sources of bias
Version: 1Date:24 September 2010
Reviewer: Nynke Smidt
Reviewer's report:
Title manuscript: Methodological quality of test accuracy studies included in
systematic reviews in obstetrics and gynaecology: sources of bias.
The results described in this manuscript are based on 300 diagnostic test
accuracy studies included in 10 systematic reviews which were performed in
period of 2004-2007. The authors a) investigated the methodological quality of
the diagnostic test accuracy studies using QUADAS, b) evaluated the quality of
reporting of the diagnostic test accuracy studies using the STARD checklist, c)
compared the methodological quality in studies published before and after the
introduction of QUADAS, d) compared the differences in the methodological
quality between studies in obstetrics and gynaecology, e) investigated whether
the methodological quality was associated with study size and geographical
areas, f) investigated which items of the QUADAS list was associated with bias.
This manuscript is very close related to the manuscript ‘The quality of reporting of
primary test accuracy studies in Obstetrics and Gynaecology: application of the
STARD criteria’. Both manuscripts are based on the same reviews and studies
and used the same methodology.
Minor issues not for publication
None
Discretionary revisions
1. Methods: Please, check the methods and results of the statistical analysis of
the data with a statistician.
2. Overall: Although, I am not a native speaker, but to my opinion the text needs
some editing (for example ‘In those studies where they this was not done there
was an overestimation in the accuracy of the test’).
3. Results: it would be interested to see how many studies have included a
flowchart and whether this is associated with better methodological quality.
4. Results: I would remove Figure 2 from manuscript, as it takes a lot of space
and only provides a little bit information. Maybe this information can be presented
in the text or table (with providing other information).
5. Results: it would be interested to see what kind of articles are included (i.e.
design characteristics (case-control versus cohort studies), what kind of journals,
sample size of studies, test characteristics (index and reference standard), target
conditions, study population, setting etc.
Minor essential revisions
6. Figure 1 is difficult to read. Numbers are too small and differences between
yes, no, unclear and not applicable are not presented clearly.
7. Figure 3. Please, provide the explanation of the QUADAS items (in the figure
or legends).
Major compulsory revisions
8. Methods: Please, present how you have selected the systematic reviews
(search and selection criteria)? Did the authors include all studies included in the
systematic reviews or did the authors have selection criteria for these studies too
(ie. Minimum sample size)?
9. Results: The authors discuss the QUADAS and STARD list in general. It would
be interested to see which items are poorly reported and cause bias? Are there
differences between the 2 specialties? How can these differences be explained?
Did the authors found the same poorly reported items compared to other reviews
investigated the quality of reporting using STARD?
10. Discussion: Can the authors make recommendations for authors with regard
to the individual poorly reported STARD items?
11. Methods: Why are the authors interested in the differences between
countries? Why would the authors expect a difference? I do not understand the
rationale behind the following sentence “Countries were grouped depending on
the number of articles published and the mean journal impact factor and adjusted
for gross domestic product and population”. Please, can this be explained? The
authors make a firm statement in the abstract about the positive correlation
between the geographical area of publication and the compliance with QUADAS
(only for gynaecology). Before the authors make this statement, would be
possible to explain the differences between the results of the primary analysis
and the sensitivity analysis?
12. Methods: I am wondering whether the authors have adjusted the QUADAS
list to the topic of the systematic review. For example, item 3 (reference
standard). How did the authors deal with this?
Level of interest: An article of importance in its field
Quality of written English: Needs some language corrections before being
published
Statistical review: Yes, but I do not feel adequately qualified to assess the
statistics.
Declaration of competing interests:
'I declare that I have no competing interests'