2009), a study of flipchart provision (Glewwe et al. 2004), and a study of student incentives (Kremer, Miguel, & Thornton 2009), all in Kenya.

One natural explanation for the difference in composition is the inclusion rules of the reviews. Conn (2014) only includes studies in Sub-Saharan Africa, for example. However, if one looks at the studies that are included in all but one of the reviews, allowing for the possibility that many studies may be included in all reviews except Conn (2014), one finds only 4 studies (again, out of a total of 227). If one expands the field to studies included in most reviews (4, 5, or all 6), a total of only 13 studies achieve this (6 percent of the sample). Some of this exclusion may also be based on measures of quality, whether subjective or based on explicit criteria – e.g., Murnane and Ganimian (2014) exclude studies that rely exclusively on fixed effects or matching methods. While some exclusions are justified by explicit search restrictions, many are not. To illustrate this point we contrast two studies. The first, Angrist & Lavy (2001), is an evaluation of a teacher training intervention in Israel, evaluated using a matching strategy. The second, Gee (2010) is an RCT of an anti-malarial program with learning outcomes in Kenya. We can easily see why Angrist & Lavy (2001) would be excluded from Conn (2014) and Murnane and Ganimian (2014); the former focuses on Sub-Saharan Africa and the latter explicitly excludes matching studies. Gee (2010), however, falls into the common denominator of inclusion criteria across almost all reviews: it is an RCT with learning outcomes at the primary school level, in Sub-Saharan Africa, published between 1990 and 2010. According to the stated search strategies of the reviews therefore, there is no reason why it should be excluded from any review except Krishnarate, White, & Carpenter (2013), who cut off their search in 2009. In practice, however, it is only included in Conn (2010).

Other distinguishing inclusion criteria include that two reviews only include RCTs, whereas the other four include RCTs and studies using quasi-experimental methods. However, even with randomized trials the overlap in studies is limited (Table 8 and Figure 4). Of 134 learning RCTs, over half (74 studies) are included in only one review. As with the wider collection of learning studies, only 13 studies are included in most (4, 5, or 6) of the reviews. The largely non-overlapping collection of studies is apparently driven neither by geography nor by methodology.

While there are differences in the scope of each review, we consider each inclusion criterion common to all reviews, successively - learning outcomes, primary school, RCTs, the 1990-2010 time frame, and implementation in Sub-Saharan Africa – to examine how much of the variation in composition is driven by inclusion criteria (Table 8). Across the overall sample of 301 studies, 227 look at learning outcomes. Coverage of these learning studies in any single review is low, ranging from 30 studies (13 percent) in Kremer, Brannen, & Glennerster (2013) to 92 studies (41 percent) in Murnane & Ganimian (2014). To account for the fact that this might be driven by inclusion restrictions on methodology, we next consider the 134 RCTs with learning outcomes. Among these, coverage in any single review is even lower, ranging from 12 studies (9 percent) in Glewwe et al. (2014) to 68 studies (51 percent) in Murnane & Ganimian (2014). We next add restrictions for studies which include primary level outcomes and which were published between 1990 and 2010. Of the 107 studies fulfilling all of the above requirements, only between 11 percent and 60 percent of studies are included in any single review.

8