DOI: 10.5061/dryad.5f4g6
title: Data from: Correction for bias in meta-analysis of little-replicated studies

description: 1. Meta-analyses conventionally weight study estimates on the inverse of
 their error variance, in order to maximize precision. Unbiased variability
 in the estimates of these study-level error variances increases with the
 inverse of study-level replication. Here we demonstrate how this
 variability accumulates asymmetrically across studies in
 precision-weighted meta-analysis, to cause undervaluation of the
 meta-level effect size or its error variance (the meta-effect and
 meta-variance). 2. Small samples, typical of the ecological literature,
 induce big sampling errors in variance estimation, which substantially
 bias precision-weighted meta-analysis. Simulations revealed that biases
 differed little between random- and fixed-effects tests. Meta-estimation
 of a one-sample mean from 20 studies, with sample sizes of 3 to 20
 observations, undervalued the meta-variance by ~20%. Meta-analysis of
 two-sample designs from 20 studies, with sample sizes of 3 to 10
 observations, undervalued the meta-variance by 15-20% for the log response
 ratio (lnR); it undervalued the meta-effect by ~10% for the standardised
 mean difference (SMD). 3. For all estimators, biases were eliminated or
 reduced by a simple adjustment to the weighting on study precision. The
 study-specific component of error variance prone to sampling error and not
 parametrically attributable to study-specific replication was replaced by
 its cross-study mean, on the assumption of random sampling from the same
 population variance for all studies, and sufficient studies for averaging.
 Weighting each study by the inverse of this mean-adjusted error variance
 universally improved accuracy in estimation of both the meta-effect and
 its significance, regardless of number of studies. For comparison,
 weighting only on sample size gave the same improvement in accuracy, but
 could not sensibly estimate significance. 4. For the one-sample mean and
 two-sample lnR, adjusted weighting also improved estimation of
 between-study variance by DerSimonian-Laird and REML methods. For
 random-effects meta-analysis of SMD from little-replicated studies, the
 most accurate meta-estimates obtained from adjusted weights following
 conventionally-weighted estimation of between-study variance. 5. We
 recommend adoption of weighting by inverse adjusted-variance for
 meta-analyses of well- and little-replicated studies, because it improves
 accuracy and significance of meta-estimates, and it can extend the scope
 of the meta-analysis to include some studies without variance estimates.
description: Data
 S1R
 script for simulations. Simulations of fixed- and random-effects
 meta-analysis using alternative estimators: one-sample mean, two-sample
 Hedges' g, and two-sample lnR, for comparison of performance by
 inverse-variance weighting and inverse-adjusted-variance
 weighting.Doncaster&Spake_Data_S1.txtData S2R script for calculating mean-adjusted error variance. Finds the mean-adjusted study variance for all the primary studies contributing to a meta-analysis, for a one-sample mean, or two-sample log response ratio, or or two-sample Hedges' g.Doncaster&Spake_Data_S2.txt