ANALYTIC STRATEGY: Systematic review and meta-analysis of intergroup interventions to reduce violent intergroup conflict and promote post-conflict reconciliation
Creators: name: Stefaniak, Anna
name: Halperin, Eran
name: Atia, Renana
name: Hameiri, Boaz
name: Hebel, Shira
name: Nir, Nimrod
name: Ruhrman, Anat
name: Shulman, Deborah
name: Shuman, Eric
name: Testerman, Meghan
name: Vandermeulen, Daan
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: Social Psychology
subject: Social and Behavioral Sciences
subject: Psychology
subject: FOS: Psychology

description: 1. Title Systematic review and meta-analysis of intergroup interventions to reduce violent intergroup conflict and promote post-conflict reconciliation 2. Keywords Conflict reduction; intergroup conflict; intergroup intervention; personalization parameters; peace; meta-analysis 3. Review &amp; meta-analysis milestones Completed tasks (completion date): Defining main terms &amp; goals of the search (February 1, 2021) Creating a reference set (April 30, 2021) Study inclusion/exclusion criteria and design (June 15, 2021) Creating a list of search terms (July 7, 2021) Systematic Search (July 29 &amp; August 30, 2021) Planned tasks (planned completion date): Screening &amp; data Extraction (November 30, 2021) Quality Assessment of Data; Analysis of Data (December 31, 2021) Project Write-Up (February 28, 2022) 4. Stage of review at time of this submission Preliminary searches have been completed, the main database search has been conducted, and the dataset has been manually de-duplicated. No screening or analysis of the dataset has been conducted. 5. Research team PI: Dr. Anna Stefaniak, anna.stefaniak@carleton.ca, Carleton University Prof. Eran Halperin, eran.halperin@mail.huji.ac.il, Hebrew University of Jerusalem Renana Atia, renana.atia@mail.huji.ac.il, Hebrew University of Jerusalem Boaz Hameiri, bhameiri@tauex.tau.ac.il, Tel Aviv University Shira Hebel, shira.hebel@mail.huji.ac.il, Hebrew University of Jerusalem Nimrod Nir, anisker@gmail.com, Hebrew University of Jerusalem Anat Ruhrman, anat_ruhr@yahoo.com, Hebrew University of Jerusalem Deborah Shulman, dshulman23@gmail.com, Hebrew University of Jerusalem Eric Shuman, eshuman1650@gmail.com, Hebrew University of Jerusalem Meghan M. Testerman, mtesterman@princeton.edu, Princeton University Daan Vandermeule, daan.vandermeule@mail.huji.ac.il, Hebrew University of Jerusalem 6. Conflicts of interest Members of the research team declare that they have no conflict of interests. 7. Review question Main question: Are intergroup interventions, that is evidence-based actions aimed at reducing intergroup conflict and hostility (see Halperin &amp; Schori-Eyal, 2020), effective at promoting peace in ongoing violent intergroup conflict and post-conflict societies? Secondary question: Are there moderators of the effectiveness of intergroup interventions to promote peace? We are particularly interested in moderators that can constitute personalization parameters (as described in Halperin &amp; Schori-Eyal, 2020), that is person-level characteristics that determine for whom a given type of intervention is most (or least) effective. Examples of personalization parameters include political orientation, personality, and dominant emotional reactions to conflict (see Halperin &amp; Schori-Eyal, 2020). 8. Searches Database Searching • We will search the following databases: APA PsycINFO (EBSCO) and Science Citation Index Expanded, Social Sciences Citation Index and Arts and Humanities Citation Index (Web of Science), ProQuest Dissertations &amp; Theses Global • The search will be conducted in the title and abstract fields of the databases. • There will be no limitation on publication date or language in the search query although where appropriate subject categories may be used to limit the search results. • Exact search syntax, number of results, and date of search will be provided. • An additional database search will be conducted before the final analysis in order to capture any recent publications that meet the inclusion criteria. Additional sources of published and unpublished literature • In addition to the database searches, we will include articles identified by the research team as the “reference set” (i.e., a set of articles that meet all search criteria), as well as search the reference lists of those articles. We will also reach out to experts in the field using professional associations’ mailing lists. Specifically, we will reach out to members of the Society of Personality and Social Psychology, International Society of Political Psychology, and European Association of Social Psychology. • We will try to access unpublished studies as well by reaching out to researchers as well as practitioners (e.g., non-profit organizations) working to promote peace. 9. URL to search strategy. Search strategy link: https://drive.google.com/drive/folders/1AD7cGxsNTjsJrBTGF1kv7kFi25zvNe6i 10. Condition or domain being studied. This work focuses on interventions to promote peace, that is on evidence-based actions aimed at promoting intergroup peace and/or reconciliation among social groups that used to be or are currently engaged in a violent conflict. 11. Participants/population. Population: People in societies with ongoing or resolved violent intergroup conflict When reviewing the reference set of articles we realized that in some instances abstracts and/or titles do not mention the word “conflict” (or related keywords) explicitly but rather talk about the specific conflict context (e.g., Israel, Rwanda, etc.) we decided to include a list of known conflict settings in our search to make it more comprehensive. To this effect, we created a list of countries with recorded violent intergroup conflict that occurred between 1946 and 2020. The list was based on two lists of conflicts: 1) the Major Episodes of Political Violence, 1946-2018 database created by Center for Systemic Peace (CSP) and accessible through their website (http://www.systemicpeace.org/inscrdata.html) and (2) the UCDP/PRIO Armed Conflict Dataset version 21.1 accessible from the Uppsala Conflict Data Program (UCDP) &amp; Peace Research Institute Oslo (PRIO) (https://ucdp.uu.se/downloads/). The two datasets were used to provide a more comprehensive coverage as they differ slightly in their definition of violent conflict. The CSP defines armed conflict as “[…] systematic and sustained use of lethal violence by organized groups that result in at least 500 directly-related deaths over the course of the episode.” The definition can be found in the codebook for the Major Episodes of Political Violence, 1946-2018 database available here: http://www.systemicpeace.org/inscrdata.html The UCDP defines armed conflict as “[…] a contested incompatibility that concerns government and/or territory where the use of armed force between two parties, of which at least one is the government of a state, results in at least 25 battle-related deaths in one calendar year.” (https://www.pcr.uu.se/research/ucdp/definitions/#tocjump_8252132865605607_9). 12. Intervention(s), exposure(s). Please note that the following were not included as search terms in our search strategy in order to avoid missing studies that utilized less-established interventions that are not included in the above (extensive) list. Intergroup interventions to promote peace, as defined above (and in Halperin &amp; Schori‐Eyal, 2020), may utilize – but are not limited to – the following methods of intergroup attitude/behavior change: Intergroup contact (direct, extended, imagined, vicarious, para-social; see Pettigrew &amp; Tropp, 2011); Paradoxical thinking (Hameiri et al., 2018); Group malleability (Halperin et al., 2011); Outgroup criticism (McDonald et al., 2018); Social norms/ Peer influence (Perkins et al., 2011); Collective blame (dissonance) (Bruneau et al., 2020); Self-affirmation (Čehajić-Clancy et al., 2011); Moral exemplars (Čehajić-Clancy &amp; Bilewicz, 2021); Empathy/perspective taking (Galinsky &amp; Moskowitz, 2000); Cognitive &amp; emotional training (Halperin et al., 2014); Social categorization (Gaertner et al., 1993); Discussion/dialogue (Gurin et al., 1999); Value consistency and self-worth (Rokeach, 1971); Multicultural/anti-bias education (Lustig, 2003); Entertainment (Bilali &amp; Vollhardt, 2013); Diversity training and cross-cultural/intercultural training (Hanover, &amp; Cellar, 1998); Cooperative learning (Roseth et al., 2008). 13. Comparator(s)/control. We will only include studies that utilize experimental design with a non-exposed control group. 14. Types of study to be included. We will only include experimental studies that randomly assign participants to the treatment (intervention) and control (no exposure to intervention) conditions, specifically, we will only include: • Randomized posttest only with control (POWC) • Pretest–posttest with control (PPWC) Studies conducted in the laboratory, in the field, and online will all be included (though we will code for these characteristics and analyze whether they bear on the effects of the interventions). 15. Main outcome(s). Main outcome variable: support for peace Support for peace is defined as positive changes in attitudes/behavior intentions/behavior towards the relationship with the adversary* (e.g., forgiveness, willingness to reconcile, offering/supporting apologies) and towards the conflict (e.g., support of group-level [macro-level] actions that explicitly promote peaceful resolution and/or reconciliation, support for peace, etc.). Change is defined as the difference between the control (baseline) group and the experimental group(s) and/or as the difference in the size of change observed between pre- and post-treatment level of dependent variable(s) in the experimental as compared to the control condition. Not included are attitudes and behaviors towards the adversary that are not specifically related to the context of conflict (e.g., prejudice and stereotypes of the adversary, which may well exist regardless of the conflict). Additionally, we will analyze personalization parameters (i.e., moderators of the effectiveness of interventions to promote peace) such as political orientation; personality; dominant emotions; group identification (as described in Halperin &amp; Schori‐Eyal, 2020) to establish whether certain interventions are more effective for participants with certain characteristics as well as more typical moderators of effectiveness (e.g., study design, methodological characteristics). 16. Inclusion and exclusion criteria The definition of the outcome measures, the interventions of interest, methodological criteria, and the context of the studies of interest (i.e., violent intergroup conflict ongoing or resolved) are a source of our theoretical and methodological inclusion and exclusion criteria. Included will be: • experimental studies that utilize pretest-posttest with a control group or posttest only with control design (i.e., random allocation to intervention and control group[s]); and • studies that assess at least one dependent variable that meets the outcome variable definition criteria; and • studies conducted in the context of violent intergroup conflict * Should some of the identified studies that meet these criteria report data from clusters (e.g., classrooms) assigned to conditions and the number of clusters is low (i.e., &lt; 10), we may need to exclude these from analyses, due to difficulties in estimating unbiased standard errors and potential bias in estimators (see the Supplement to the Paluck et al., 2021). Excluded will be: • studies that evaluate attitudes towards the adversary in conflict; or • non-experimental studies (e.g., correlational, pretest-posttest only); or • studies of conflict that is not violent (e.g., conflict in organizations) In addition, we will limit our analysis to published and unpublished reports in English. 17. Data extraction (selection and coding). Each article will be coded by two members of the research team (who are all PhD students, post-docs, or faculty members), independently of each other, and blind to the others’ coding. Any discrepancies will be discussed to arrive at an agreed-upon coding. This screening and coding of articles will be done in three stages: (1) Stage 1: screening abstracts &amp; titles Articles will be screened based on the title and abstract to remove non-relevant entries (this will be performed by one person per article, but the coders will be instructed to err on the side of caution and include all possibly relevant articles) (2) Stage 2: screening full-text At this stage full-text screening will be performed (two coders per article) to ensure that only relevant articles that meet all inclusion criteria are left (3) Stage 3: coding Two coders will extract all relevant information from the articles identified at Stage 2. Any discrepancies will be resolved through documented discussion between two coders; if agreement could not be reached, the PI will weigh in on the decision. All extracted data will be recorded in a google doc spreadsheet file shared by the team. Data extraction • Participants’ data: - age (average in a given study); - ethnicity - group belonging (i.e., membership in one of the groups in conflict) - location of the study (e.g., school, workplace, community, university, online) will be recorded for study participants. • Study design data: - # of the study in the paper (e.g., 1/2/3 of 5) - number of participants in the intervention group - number of participants in the control group - type of experiment (laboratory/field/online [or survey]); - type of publication (e.g., peer-reviewed article, book chapter, dissertation/thesis, report); - type of control group: empty (no treatment)/ waitlist/placebo); - type of outcome (attitude/behavior intentions/behavior) - focus of outcome (relationship with the adversary [e.g., offer of forgiveness] vs. conflict related [e.g., policy support]) - type of intervention (coded for all interventions listed in #12 or “other”); - intensity/length of intervention (brief [under 30 minutes] vs. longer [30+ minutes]); - number of intervention sessions; - type of measurement delay (immediate measurement/immediate + delayed measurement/only delayed measurement); - length of delay (in days) - preregistration status (yes/no); - open science status (data/materials available: yes/no) • Effects - effect type (difference in means; difference in difference [the difference between changes in the experimental and control groups]; Cohen’s d; regression coefficient; t-test; F-test; eta-squared; partial eta-squared) (*note that all these are included here because we will be extracting any data that allows to calculate effect size, should the actual effect size not be provided by the authors) - raw statistics (are all raw statistics provided to calculate effect size: yes/no) - personalization parameters (are personalization parameters of interest reported: yes/no; if no – reach out and ask about relevant measures; if yes – reach out and ask for data) - location of relevant information in the article (page number) If relevant information is not provided in a given source, we will contact study author(s) to try and extract it directly from them. 18. Risk of bias (quality) assessment. As recommended by Vevea et al. (2019) we will combine several methods that assess quality and publication bias. Should publication bias be detected the final results will be adjusted accordingly. During data extraction, coders will conduct a critical appraisal of the included studies, drawing on relevant bias domains from ‘The Cochrane Collaboration’s tool for assessing risk of bias’ - randomization of participants (in experimental studies) and assessment of self selection (akin to blinding participants) (Higgins &amp; Altman, 2008). We will create a funnel plot to assess the relation between sample size and effect size, estimate a p-curve (Simonsohn et al., 2014), and conduct Egger’s test of the intercept (should we find a sufficient number of studies to be evaluated). We will also perform subgroup analysis by source (published vs. unpublished source), and study design characteristics (e.g., lab/field/online studies). 19. Strategy for data analysis. In this analysis we will utilize random effects approach and we will compare the control group(s) (treated as baseline) to treatment (i.e., intergroup intervention) group(s) (or, in case of pretest-posttest with control, the change in the treatment vs. in control condition). We expect to see an increase in support for peace (as defined in #15) in the treatment groups that have been exposed to intergroup interventions. We will calculate Hedge’s g coefficient to gauge the effect size in each of the included studies. This effect will be averaged for up to 5 most important outcome variables (that meet the dependent variable definition provided herein) reported at the study level. ANALYTIC PLAN (May 2022 update; prior to seeing the data): Based on Borenstein et al., (2009); Harrer et al. (2021); Konstantopoulos and Hedges (2019). a) General analytic strategy In this meta-analysis we are interested in gauging the effectiveness of interventions to promote peace in violent intergroup conflict (ongoing or resolved). We are only including experimental studies with random allocation to control and experimental conditions. The main comparison of interest is between intervention (treatment) and empty control conditions, however, we will also include effects calculated using active control groups (i.e., non-empty controls). The type of control group will be recorded and its effects analyzed in subgroup analysis. The main analysis will use random effects model to calculate the overall effect of intergroup interventions on support for peace across all relevant studies. For each study we will code up to five most important dependent variables (see Paluck et al., 2021 for a similar approach). Their effects and sampling variances will be averaged and the average values will be used as input in the meta analysis. We will use Hedge’s g coefficient to gauge the effect size in each of the included studies. Tau-squared will be calculated using the Restricted Maximum Likelihood (REML) estimator. We will also employ the Knapp-Hartung adjustment to reduce the chance of false positives. Note on exclusions. Studies for which it will be impossible to find/reconstruct treatment effects and whose authors do not provide sufficient data upon personal request will be excluded from analyses. b) Controlling for clustering of effects from the same paper and analysis of outliers and influential cases To check robustness of our findings we will recalculate our analyses while controlling for clustering of the effects by paper (using a multi-level meta-analysis). We will also perform an analysis of outliers using simple outlier identification (i.e., studies whose effect size confidence interval does not overlap with the pooled effect size confidence interval will be temporarily excluded and pooled effect recalculated) and analysis of influential cases using the leave-one-out method. c) Assessment of publication bias We will assess publication bias using the following methods: Funnel plot and quantified with the Egger’s Regression Test (Egger et al,, 1997). The trim and fill method (Duval &amp; Tweedie, 2000) that allows to check how robust the initial findings are against the publication bias. Note though that the trim-and-fill method is not suitable for high heterogeneity among studies and that under high heterogeneity its results may be unreliable. If appropriate (i.e., if the number of studies in our meta-analysis is &gt; 20 and the heterogeneity is below &lt; 80%), we will also use the PET-PEESE method (Stanley 2017) to identify the true effect size. If the heterogeneity of identified studies is small to moderate (I2 &lt; 50%), we will also use the p-curve analysis to verify whether there is evidence for the existence of a non-zero intervention effect, given the p-values obtained in the studies and we will also use p-curve analysis to estimate the true effect size in our analysis (for higher heterogeneity such analyses may be performed in subgroups). . d) Subgroup analysis and meta-regression Next, we will conduct subgroup analysis and meta-regression analyses to assess how the following characteristics of the studies shape relevant effect sizes. Please note that we will only perform these analyses if there are at least 10 studies that provide us with variables of interest: Methodological factors: 1) Pre-registration status (Yes - No); note that it will be assumed that studies that do not explicitly report having been preregistered will be coded as not pre-registered 2) Intervention type (Lab - Field - Survey/Online) 3) Type of control group (empty vs. active vs. comparison between two interventions) 4) Intervention length 5) Outcome type (Attitude - Behavior intentions – Behavior – Physiological) 6) Short vs. Long term effects (we will compare immediate measurement to measurement with any kind of delay; as well, we will use the following categorization scheme to assess whether the length of delay is consequential for the magnitude of effects: 0 = immediate; 1 = up to one week; 2 = week to a month; 3 = a month and a day to 6 months; 4 = 6 months and a day-12 months; 5 = year and a day or more 7) Synergy (Only one intervention used – More than 1 interventions combined) Contextual factors: 1) Situation (Active conflict – Post-conflict), as defined the presence of a peace agreement/resolution and meeting the definition of a violent conflict by the Uppsala Conflict Data Program (UCDP) &amp; Peace Research Institute Oslo (PRIO) (https://ucdp.uu.se/downloads/). 2) Target group (majority/advantaged/aggressor - minority/disadvantaged/victim group) 3) Type of intervention – given that we cannot know how many studies in the dataset used different types of interventions it is impossible to know how many will have been used a sufficient number of times to warrant comparisons; thus the exact type of interventions to be compared will be determined by the types of interventions used in relevant studies 4) Platform (Education - Media campaign - Newspaper article – Workshop - Other) 5) Typology of intervention - we will categorize all identified interventions as either providing new information, new experiences, or new skills to participants (i.e., we will use the typology provided by Hameiri et al., 2014) and analyse whether the category of intervention moderates the effect size. e) Analysis of personalization parameters For exploratory purposes we will investigate personalization parameters (Halperin &amp; Schori-Eyal, 2020) to verify whether some interventions are more effective (i.e., lead to stronger positive changes) for participants with particular characteristics? Specifically, as part of full-text coding, the research team identified the most commonly assessed characteristics of participants in the included articles. These were: political orientation; age; gender; religiosity; social identification. As part of the present meta-analysis we will investigate the moderating effects of these personalization parameters on the effects of the three types of intergroup interventions identified by Hameiri et al. (2014). As with the subgroup and meta-regression analyses these will only be performed if at least 10 studies include variables of interest. 20. Analysis of subgroups. We will conduct analyses in subgroups (a subgroup needs to consist of at least 3 studies to be considered) based on the type of outcome measured (attitudes vs. behavioral intentions vs. behavior); type of experiment (lab vs. field vs. online/survey experiment); type of intervention (this will be determined based on the findings as at this point it is not known which intergroup interventions will be represented in the identified literature); and type of setting (e.g., university/community/workplace/online etc. as well as conflict setting: Israel/Northern Ireland/Cyprus/South Africa/etc., depending on how many studies are identified pertaining to different conflict regions). These analyses will be useful in determining characteristics of the intervention and the context most likely to yield significant effects. We will test for heterogeneity in these models using the analyses of variance for effect sizes procedures described in Konstantopoulos and Heghes (2019). In order to analyze the role of personalization parameters in shaping the effectiveness of intergroup interventions to promote peace we will include the relevant personalization parameters (e.g., political orientation, personality, dominant emotions) as predictors of effect sizes in multiple regression for effect sizes models (see Konstantopoulos &amp; Hedges, 2019) and/or conduct subgroup comparisons (where appropriate). 21. References: Bilali, R., &amp; Vollhardt, J. R. (2013). Priming effects of a reconciliation radio drama on historical perspective-taking in the aftermath of mass violence in Rwanda. Journal of Experimental Social Psychology, 49(1), 144-151. https://doi.org/10.1016/j.jesp.2012.08.011 Borenstein, M., Hedges, L. V., Higgins, J. P., &amp; Rothstein, H. R. (2021). Introduction to meta-analysis. John Wiley &amp; Sons. Bruneau, E. G., Kteily, N. S., &amp; Urbiola, A. (2020). A collective blame hypocrisy intervention enduringly reduces hostility towards Muslims. Nature Human Behaviour, 4(1), 45–54. https://doi.org/10.1038/s41562-019-0747-7 Čehajić‐Clancy, S., &amp; Bilewicz, M. (2021). Moral exemplar intervention: A new paradigm for conflict resolution and intergroup reconciliation. Current Directions in Psychological Science. https://doi.org/10.1177/09637214211013001 Čehajić-Clancy, S., Effron, D. A., Halperin, E., Liberman, V., &amp; Ross, L. D. (2011). Affirmation, acknowledgment of in-group responsibility, group-based guilt, and support for reparative measures. Journal of Personality and Social Psychology, 101(2), 256–270. https://doi.org/10.1037/a0023936 Gaertner, S. L., Dovidio, J. F., Anastasio, P. A., Bachman, B. A., &amp; Rust, M. C. (1993). The common ingroup identity model: Recategorization and the reduction of intergroup bias. European review of social psychology, 4(1), 1-26. https://doi.org/10.1080/14792779343000004 Galinsky, A. D., &amp; Moskowitz, G. B. (2000). Perspective-taking: decreasing stereotype expression, stereotype accessibility, and in-group favoritism. Journal of Personality and Social Psychology, 78(4), 708-724. https://doi.org/10.1037/0022-3514.78.4.708 Gurin, P., Peng, T., Lopez, G., &amp; Nagda, B.R. (1999). Context identity and intergroup relations. In: D. Prentice &amp; D. Miller (Eds.) Cultural divides: The social psychology of intergroup contact (pp. 133–170). Sage. Halperin, E., Pliskin, R., Saguy, T., Liberman, V., &amp; Gross, J. J. (2014). Emotion regulation and the cultivation of political tolerance: Searching for a new track for intervention. Journal of Conflict Resolution, 58(6), 1110-1138. https://doi.org/10.1177/0022002713492636 Halperin, E., Russell, A. G., Trzesniewski, K. H., Gross, J. J., &amp; Dweck, C. S. (2011). Promoting the Middle East peace process by changing beliefs about group malleability. Science, 333(6050), 1767–1769. https://doi.org/10.1126/science.1202925 Halperin, E., &amp; Schori‐Eyal, N. (2020). Towards a new framework of personalized psychological interventions to improve intergroup relations and promote peace. Social and Personality Psychology Compass, 14(5), 255-270. https://doi.org/10.1111/spc3.12527 Hameiri, B., Nabet, E., Bar-Tal, D., &amp; Halperin, E. (2018). Paradoxical thinking as a conflict-resolution intervention: Comparison to alternative interventions and examination of psychological mechanisms. Personality and Social Psychology Bulletin, 44(1), 122-139. https://doi.org/10.1177/0146167217736048 Hanover, J. M., &amp; Cellar, D. F. (1998). Environmental factors and the effectiveness of workforce diversity training. Human Resource Development Quarterly, 9(2), 105-124. https://doi.org/10.1002/hrdq.3920090203 Harrer, M., Cuijpers, P., Furukawa, T.A., &amp; Ebert, D.D. (2021). Doing Meta-Analysis with R: A Hands-On Guide. Boca Raton, FL and London: Chapmann &amp; Hall/CRC Press. Higgins, J., &amp; Altman, D. G. (2008). Assessing risk of bias in included studies. In: J. Higgins &amp; S. Green (Eds.) Cochrane Handbook for Systematic Reviews of Interventions, Version 5.0.0 (pp. 187-241). John Wiley &amp; Sons. Konstantopoulos, S. &amp; Hedges, L. V. (2019). Statistically analyzing effect sizes: Fixed- and random-effects models. In: H. Cooper, L. V. Hedges, &amp; J. C. Valentine (Eds.) The handbook of research synthesis and meta-analysis (3rd edition, pp. 245-279). Russell Sage Foundation. Lustig, I. (2003). The influence of studying foreign conflicts on students’ perceptions of the Israeli Palestinian conflict. Unpublished master’s thesis: University of Haifa, Israel McDonald, M., Brindley, S., Halperin, E., &amp; Saguy, T. (2018). Outgroup members’ internal criticism promotes intergroup openness: the role of perceived risk. British Journal of Social Psychology, 57(1), 95-111. https://doi.org/10.1111/bjso.12231 Paluck, E. L., Porat, R., Clark, C. S., &amp; Green, D. P. (2021). Prejudice reduction: Progress and challenges. Annual Review of Psychology, 72(1), annurev-psych-071620-030619. https://doi.org/10.1146/annurev-psych-071620-030619 Perkins, H. W., Craig, D. W., &amp; Perkins, J. M. (2011). Using social norms to reduce bullying: A research intervention among adolescents in five middle schools. Group Processes &amp; Intergroup Relations, 14(5), 703-722. https://doi.org/10.1177/1368430210398004 Pettigrew, T. F., &amp; Tropp, L. R. (2013). When groups meet: The dynamics of intergroup contact. Psychology Press. Rokeach, M. (1971). Long-range experimental modification of values, attitudes, and behavior. American Psychologist, 26(5), 453-459. https://doi.org/10.1037/h0031450 Roseth, C. J., Johnson, D. W., &amp; Johnson, R. T. (2008). Promoting early adolescents' achievement and peer relationships: The effects of cooperative, competitive, and individualistic goal structures. Psychological bulletin, 134(2), 223-246. https://doi.org/10.1037/0033-2909.134.2.223 Simonsohn, U., Nelson, L. D., &amp; Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143(2), 534-547. https://doi.org/10.1037/a0033242 Vevea, J. L., Coburn, K., &amp; Sutton, A. (2019). Publication bias. In: H. Cooper, L. V. Hedges, &amp; J. C. Valentine (Eds.) The handbook of research synthesis and meta-analysis (3rd edition, pp. 383-429). Russell Sage Foundation.