It took 13.2 (95% CI: 12.2, 14.2) minutes to complete use of AMSTAR for each review, while it took less time to complete scoring of OQAQ, averagely 9.3 (95% CI: 8.8, 9.9) minutes per review (paired difference = 3.9, P < 0.001). Besides, a linear regression analysis was performed (time = 6.35 + 2.94 × langrage + 1.75 × log(pages), P < 0.001), revealed the time needed to complete using AMSTAR had significant associations with logarithm of the number of pages (unstandardized coefficients = 1.75, 95% CI: 0.61 to 2.90) and langrage (unstandardized coefficients = 2.94, 95% CI: 0.73 to 5.15); that is, systematic reviews with more pages or written in English need more time to complete scoring. The two assessors found there's difficulty in approaching a final decision on item 9 "were the methods used to combine the findings of studies appropriate" and item 5 "was a list of studies (included and excluded) provided"; for the latter, more detailed guidance for scoring "yes" are required.