# 3.3 Main Results

Table 4 shows the simple OLS regressions for three different specifications.<sup>12</sup> The OLS regressions register r-square of around 0.40 indicating acceptable fit of the variables explaining the outcomes and, in particular, both OLS models show positive and significant signs for the precision variable. Costa-Font et al. (2011a) explain that when this parameter is different than zero, then there is intrinsic bias due to publication, “finding that estimates are related with their standard errors.” However, OLS results can be influenced by studies that provide greater numbers of observations (equal weight is given to all observations). OLS estimates are representative of the appropriate sample frame since the selected observations are studies, not their resultant observations.<sup>13</sup> Stanley (2006) points out that paper’s characteristics may reduce the biases when the model is re-estimated using proper metaanalysis weights. Model 3, a variation of the partial correlation coefficient affected by the estimation model specification still shows a positive and significant estimator. Interpreting these results can be misleading because we do not know the magnitude and direction of publication and other sources of bias.

Meta-analysis Figures B1 and B2 in the Annex 1 show the cumulative biases and weights from the sample. Both figures suggest that five papers contribute a larger proportion of the bias found in OLS estimations. The figures show how each paper affects meta-impact estimations. To correct for the bias the procedure estimates the weights needed to adjust regressions through a WLS in order to significantly correct such biases<sup>14</sup> (Costa-Font et al., 2011a). The effect size in Figure B1 shows that all papers have uniform distribution with exception of four papers that present standard error (S.E.) coefficients above 0.5, ranging from 0.55 to 1.40 with

> 12 Models 1 and 2 estimate the determinants of a high t-value in impact estimates by including 2 distinct specifications using paper characteristics (e.g. precision variable). Based on Bruno and Campos (2011), we assessed whether the whole sample results might be the effect of a composition of very different types of papers, programs, and countries. Model 3 runs the determinants of paper correlations based on their characteristics. It measures the degree in which papers are differentiated enough—by sample choice, type of estimator, inclusion/exclusion of control, variables definitions –to have variability and a good representation of the literature (see Figures 1 and 2).

> 13 In order to mitigate the influence of studies where the investigators report large numbers of observations the procedure must follow WLS estimations. Weighting is done by dividing the left-hand-side and right-hand-side variables of the regression by ki, where ki denotes the number of error observations from study i. The sum of the ki for each study is one: the observations from studies that provide greater numbers of observations receive less weight in the estimation. Meta regression estimates calculate the weights adjusting publication bias as well (see Figure A2).

> 14 The advantage of WLS in meta-analysis is that it assigns larger weights to those estimates with larger precision.

22