Compared with the results reported by Shea et al. [8] , considerable differences exists either on ICC for AMSTAR contrast to the OQAQ, or on items with low kappa values. Shea et al. reported a lower ICC of 0.66 (95% CI: 0.28, 0.84), and different items (item 4 and item 7) with relatively low kappa parameter. Possible explanations for these differences include (i) the different samples of systematic reviews being evaluated, most SRs in our sample were short, published in Chinese, and more likely to be rated as low quality, such less conversely evaluations could produce a higher ICC value in our study; (ii) the extra procedure of translation and the conducts of evaluations by applying two tools simultaneously may act as a kind of consensus training to make the evaluations of AMSTAR more likely convergent with the results of OQAQ, that lead to a higher ICC value too in the present study; (iii) the background, skills, and expertise of the assessors were different from that of Shea et al. study.