DOI: 10.3886/icpsr37338.v1
title: Evaluation of the P2P Challenging Extremism Initiative, Massachusetts and Utah, 2016-2019
title: Version 1

description: This project convened experts and practitioners in the areas of program evaluation, radicalization to violent extremism, and social media analytics in order to generate and integrate scientifically derived knowledge into strategies for effective prevention and intervention against domestic radicalization and violent extremism in the United States. More specifically, we generated substantive evaluation data, which can be used by practitioners and policy makers to enhance the creation and dissemination of effective counter-narratives for reducing the threat of ideologically-motivated violence in the US. We used a mixed-methods approach to evaluate an existing nationwide initiative, Peer-to-Peer (P2P): Challenging Extremism, which aims at engaging youth in countering violent extremism in schools and online arenas. The project had four specific objectives: 1) Evaluate the content and dissemination of the P2P Initiative social media products, 2) evaluate the impact of the P2P Initiative on youth engaged in its development, 3) evaluate the impact of youth exposure to the P2P educational activities, and 4) assess the drivers of success and barriers in the implementation of the initiative. To complete these objectives, the following research phases were conducted: <strong>A secondary review</strong> of 150 P2P social media products created between fall 2015 and spring 2017, including data on end-users interactivity. <strong>Phone and in-person group interviews</strong> with faculty and students engaged in the P2P Initiative. <strong>A prospective cohort study</strong> evaluating the impact of the Kombat with Kindness (KWK) campaign on Utah secondary school students, using a pre-post intervention design. <strong>A randomized control study</strong> evaluating the impact of the Operation 250 (OP250) on Massachusetts secondary school students, using a pre-post intervention design. <strong>Phone interviews</strong> with faculty who implemented the P2P Initiative.
description: The purpose of the project was to independently evaluate an existing nationwide initiative, Peer-to-Peer (P2P): Challenging Extremism, which aims at engaging youth in countering violent extremism in schools and online arenas. By determining the reach, effectiveness, successes, and barriers of the program's implementation, practitioners and policy makers can enhance the creation and dissemination of effective counter-narratives for reducing the threat of ideologically-motivated violence in the US. The project had four specific objectives: 1) Evaluate the content and dissemination of the P2P Initiative social media products, 2) evaluate the impact of the P2P Initiative on youth engaged in its development, 3) evaluate the impact of youth exposure to the P2P educational activities, and 4) assess the drivers of success and barriers in the implementation of the initiative.
description: Web content review: 150 P2P products created 2015-2017 were categorized by campaign goals, target audience, social media metrics, and number of in-person events held. This was used to determine which P2P-created campaigns to evaluate, based on impact on target audience. Interviews with youth in the Initiative: 24 phone interviews and 4 in-person group interviews were conducted with faculty and students who were former participants in the P2P Initiative, specifically regarding their ability to develop counter-narratives or discredit extremist propaganda. Examples of propaganda narratives were obtained through web scraping of a white nationalist forum. Kombat with Kindness intervention: Researchers worked with the KWK initiative to identify schools that had implemented its materials. Two Utah middle schools were selected as the intervention condition (participated in KWK), while a third non-implementing school that was demographically similar was selected as the control condition (received materials on preparing for weather-related emergencies). 8th and 9th grade students who participated were given pretest and posttest surveys. Operation 250 intervention: Students at two Massachusetts high schools were selected for participation in the study. Participants were randomly assigned to receive educational materials related to Operation 250 (intervention condition) or to preparing for weather-related emergencies (control condition). Pretest and posttest surveys at 1 month and 2 months were distributed. Interviews with faculty implementers: Faculty and staff engaged in the P2P Initiative across the United States were recruited for phone interviews. Questions focused on reasons for joining the initiative, organizational and logistical issues, and recommendations for future initiatives.
description: In both intervention studies, items included participant demographics (age, grade level, gender, race), experiences of being treated unfairly, exposure to insulting and accepting messages, experiences of unsafe situations online, and social media use (types, frequency of use). Items unique to the KWK study were related to cultural sensitivity and empathy (e.g. I enjoy interacting with kids from different cultures), and awareness of racial/ethnic oppression. Intervention participants evaluated KWK's learning objectives; control participants were asked to list emergency supplies needed for a winter storm. Items unique to the OP250 study were related to comfort with Internet/technology use, risk assessment of online behaviors, knowledge about bias and hate, and online safety.
description: None.
description: Response Rates: Response rates ranged from 85% to 95%.
description: Presence of Common Scales: Cultural Intelligence Scale (Ang et al. 2007) Scale of Ethno-Cultural Empathy (Wang et al. 2003) Perception of racism in children and youth scale (Pachter et al. 2010)
description: Intervention study participants were recruited by schools and community-based organizations via convenience methods. OP250 study participants were randomly assigned to a condition, while KWK participants exposed to educational materials were matched with students in the control condition. Former youth participants were purposively sampled for interviews, while faculty participants who conducted phone interviews were recruited via snowball and convenience sampling.
description: Datasets: DS0: Study-Level Files DS1: Kombat with Kindness Pretest Intervention Group Data DS2: Kombat with Kindness Pretest Control Group Data DS3: Kombat with Kindness Posttest Intervention Group Data DS4: Kombat with Kindness Posttest Control Group Data DS5: Operation250 Pre-Screening Data (School 1) DS6: Operation250 Pretest Data (School 1) DS7: Operation250 1-Month Posttest Data (School 1) DS8: Operation250 Pretest Data (School 2) DS9: Operation250 1-Month Posttest Data (School 2) DS10: Operation250 2-Months Posttest Data (School 2)
description: For interviews: Youth between 14-25 years old and university faculty across the United States who participated in the P2P Initiative. For intervention studies: Youth between 14-21 years old living in Massachusetts or Utah. For web content review: Social media products created for the P2P Initiative between fall 2015 and spring 2017.. Smallest Geographic Unit: None
description: face-to-face interview; mixed mode; web scraping; on-site questionnaire; telephone interview; web-based surveyOnly the data from the cohort study and randomized control study are available at this time.