Declines in Efficacy of Anti-Bullying Programs Among Older Adolescents : Theoretical Considerations and a Three-Level Meta-Analysis

Highly visible tragedies thought to involve bullying in high schools have directly contributed to public support for state-mandated K-12 anti-bullying programming. But are existing programs actually effective for older adolescents? This paper first outlines theoretical considerations that lead to the prediction of a discontinuity in program efficacy among older adolescents. This paper then reports a novel meta-analysis of intervention programs that administered the same program to multiple age groups (k = 17, with 68 effect sizes). By conducting a hierarchical meta-analysis of the within-study moderation of efficacy by age, more precise estimates of age-related trends were possible. Results were consistent with theory in that while bullying appears to be effectively prevented in 7 th grade and below, in 8 th grade there is a sharp drop to an average of zero. There was a seeming reversal in efficacy through the high school years, such that programs, if anything, cause harm. This finding contradicts past meta-analyses that used between-study tests of moderation. Implications of these findings for developmental theory and for the interface between program evaluation research and social policy are discussed. The findings also represent a cautionary tale for the consumption of meta-analyses. Declines in Efficacy of Anti-Bullying Programs 3 Declines in Efficacy of Anti-Bullying Programs Among Older Adolescents: Theoretical Considerations and a Three-Level Meta-Analysis Bullying is an aggressive act in which one or more individuals with relatively higher social power systematically and intentionally cause harm to an individual with relatively lowerpower (Olweus, 1993). By now, the data are quite clear that victims of bullying suffer in terms of their social, emotional and academic development (Cook, Williams, Guerra, Kim, & Sadek, 2010; Reijntjes, Kamphuis, Prinzie, & Telch, 2010; Reijintjes et al., 2011; Ttofi, Farrington, Losёl, & Loeber, 2011) and even physical health (Copeland et al., 2014). These trends alone might have been cause for public action against bullying. But in the past decade and a half, there have also been a number of high-profile shootings and suicides carried out by older adolescents in high schools. Popular media interpretations have emphasized the role of bullying in these events (Grossman, April, 2, 2009; Gibbs, April, 19, 2010). It is unclear to what extent bullying actually contributes to such rare, extreme tragedies, but it is clear that such events have galvanized public support for laws requiring school-wide K-12 anti-bullying programs (Bierman, May, 4, 2010; Bullying Prevention Plan Act, Mass. Stat. c71, §§ 370, 2010). However, do existing programs work for high school students? If not, then such state laws, without more specific guidance, could lead to wasted resources or even increased bullying—at least until improved interventions were developed. Recent meta-analyses of past anti-bullying interventions have suggested that, although there is some notable variability across nations and types of programs, “on average, anti-bullying campaigns have had some modest success” (Smith, 2011, p. 419; for null effects, see Merrell, Gueldner, Ross, & Isava, 2008; for modest positive effects, see Smith, Schneider, Smith, & Ananiadou, 2004; Ttofi & Farrington, 2011). Because past meta-analyses aggregated many Declines in Efficacy of Anti-Bullying Programs 4 studies conducted with many different grade levels, it has further been possible to conduct “meta-regression” analyses to test for moderation by age. In one of the most recent and comprehensive meta-analyses, a meta-regression produced a significant positive effect of grade level (Ttofi & Farrington, 2011), leading the authors to conclude that “programs should be targeted on children aged 11 years or older rather than on younger children” (p. 46). Thus, based on the published record, policymakers may have been justified in requiring older adolescents to receive anti-bullying programs. However, these past meta-analyses may have suffered from a methodological limitation. They involved between-study comparisons of age-related effectiveness, rather than aggregating within-study tests of moderation (see Smith, Salmivalli, & Cowie, 2012; Ferguson, San Miguel, Kilburn, & Sanchez, 2007). The potential for between-study tests of moderation to mislead meta-analysts has been well-documented by statistical experts (Cooper, 2009; Cooper & Patall, 2009; also see Olkin & Sampson, 1998; Lambert, Sutton, Abrams, & Jones, 2002). Within-study analyses allow for a comparison of the same content delivered at different ages, reducing confounds and increasing precision of age-related estimates. Between-study effects however average the ages of all students in a study. For example, in a between-study moderation analysis, an intervention administered to students in 3 rd grade and 10 th grade would be treated as an effect size for 6 th grade students, even if no such students were included in the study (see, e.g., Figure 1 and row 12 of Table 2). This can mask real age-related trends. Figure 1 shows a simplified comparison to illustrate this. It shows how even though all studies may show negative slopes across grade levels, in the between-study analysis a single study with a lower average age and lower average effect size could produce a misleading, positive age-related slope. In parallel, theory and data from past research in developmental psychology supports the Declines in Efficacy of Anti-Bullying Programs 5 prediction that anti-bullying programs might be less effective in older age groups compared to younger children. There are developmental changes in the form of bullying (United States Department of Justice, 2009; 2011), the characteristics of those who bully (Cook et al., 2010), and the underlying psychological causes of bullying (Cohen & Prinstein, 2006; Cook et al., 2010; Faris & Felmlee, 2011; Guerra, Williams, & Sadek, 2011). Furthermore, there may be a domain-general developmental decline in standard techniques for behavior change, perhaps due to increases in reactance against controlling adults among older adolescents (Brehm, 1966; Erikson, 1968; Hasebe, Nucci & Nucci, 2004; Nucci, 1996; also see Larson, Wilson, Brown, Furstenberg, & Verma, 2002; Lapsley & Yeager, 2012). As we will show, much available evidence in developmental psychology would predict the exact opposite of the age trend found in Ttofi and Farrington’s (2011) meta-analysis. The present paper reviews the potential for developmental differences in the efficacy of anti-bullying programs. Part 1 reviews theoretical expectations that would lead to the prediction of declines in program effect sizes across grade levels. Next, Part 2 reports a novel metaanalysis that assesses whether the findings from past intervention studies—when analyzed using within-study moderation tests—show patterns across grade levels that conform to theoretical expectations. Finally, the paper discusses implications for future research. PART 1: THEORETICAL EXPECTATIONS