Measuring Peer Review Quality Table 1. Description of the factors that were potentially associated with quantity and impact of research publications (applicants for the LTF programme). Arithmetic mean or percent Standard deviation Minimum Maximum Variable Number of papers Decision Citations (+1) Decision Number of pages Number of co-authors Citations (+1) Decision Number of pages Number of co-authors doi:10.1371/journal.pone.0003480.t001 8.12 20% 45.97 28% 8.34 6.13 23.60 24% 9.21 6.88 Model 1: Number of papers published subsequent to application (outcome variable) Model 2: Citations for papers published prior to application (outcome variable) Model 3: Citations for papers published subsequent to application (outcome variable) 6.13 112.36 4.41 19.67 43.32 4.53 35.78 0 (rejected) 0 (rejected) 0 1 1 1 1 1 1 0 (rejected) 1 (approved) 46 1 (approved) 4,996 1 (approved) 95 663 1,123 78 2,458 calculated for the LTF applicants. The results of the regression analyses predicting number of papers (model 1) and citation counts (models 2 and 3) are presented in Table 2. We find that the number of pages per paper (see model 2) has a statistically significant influence on citation counts. In addition, we find that the coefficient for ‘‘Decision’’ is statistically significant in all three regression models. More specifically, the calculation of the percent change in expected counts [30, pp. 377–378] for a unit increase in the decision variable (from rejection to approval) following the NBRM showed that being an approved applicant increases the expected number of papers by 31%. Furthermore (see models 2 and 3), statistically significant greater numbers of citations are expected for the papers published by approved applicants prior or subsequent to applications, respectively (increased by 53% and 22%), than for the papers published by rejected applicants – holding all other variables in the models constant. Table 2. (Zero-truncated) negative binomial regression models predicting (1) number of papers published subsequent to application, (2) citations for papers published prior to application and (3) citations for papers published subsequent to application (applicants for the LTF programme). Percent change in expected counts for a unit increase in ‘‘Decision’’ with 95% confidence interval 31% [15%–50%] Note. ML-point estimates (the results of the z-test in parentheses). * p,0.05, ** p,0.01, *** p,0.001. 1truncated sample. There is one paper in the sample for model 3 with an exorbitant number of co-authors (n = 2,458) (see Table 1). Omitting this paper from the regression analysis did not alter the statistically significant coefficient for the variable ‘‘Decision’’ that is presented in the table. Interpretation example for the parameter estimates in the table: In model 2 the number of pages of a publication has a statistically significant effect on receiving citations with a parameter estimate of 0.04. This means that for an additional page, the odds of receiving citations increase by a factor of 1.04 ( = exp(0.04)), holding all other variables in model 2 constant. doi:10.1371/journal.pone.0003480.t002 PLoS ONE | www.plosone.org 5 October 2008 | Volume 3 | Issue 10 | e3480 Decision (1 = approved) Number of pages Number of co-authors Publication year Intercept npapers napplicants (clusters) Papers per applicant (cluster) Model 1: number of papers published subsequent to application 0.271*** (3.93) 2.035*** (65.29) 668 0.422*** (4.65) 0.04*** (5.60) 0.00843 (1.60) (exposure) 3,102 6521 minimum = 1 mean = 5 maximum = 28 53% [28%–82%] Model 2: citations for papers published prior to application Model 3: citations for papers published subsequent to application 24.404*** (247.17) 24.979*** (251.96) 0.196* (2.09) 0.00688 (1.14) 0.0128 (1.23) (exposure) 5,359 6451 minimum = 1 mean = 8 maximum = 46 22% [1%–46%]