Second, the study is the unit of analysis. One effect size is computed for each study. This preserves the independence of the data and gives equal weight to all included studies. Unfortunately, it also reduces the number of data points analyzed in the review. And, of course, a reviewer's biases may operate in decisions to exclude studies.
Some reviewers argue that conventional statistical tests are inappropriate for meta-analysis.
Homogeneity tests were developed to determine the likelihood that variance among effect sizes is due only to sampling error. If the homogeneity statistic is significant for a group of studies, a procedure analogous to analysis of variance can be used. Studies are repeatedly divided into subgroups according to study features until within-group variation is nonsignificant. Homogeneity tests are very likely to indicate heterogeneity among effect sizes even when the variation is of no practical or theoretical importance. Successively dividing subgroups according to these tests can capitalize on chance and cause the incorrect identification of moderators.