# Hall of Heroes: A Digital Game for Social Skills Training with Young Adolescents

Authors: Melissa E. DeRosier; James M. Thomas
Journal: International Journal of Computer Games Technology
 (2019)

---

## Abstract

Traditional social skills training (SST) programs are delivered in person and suffer from significant time, financial, and opportunity barriers that limit their reach and potential benefits for youth. This paper describes the design and preliminary evaluation ofHall of Heroes, a digital game that presents SST through an engaging superhero-themed virtual story world. Participants were randomly assigned to complete the digital game (n = 15) or to a waitlist control condition (n = 14) and were compared on parent-report measures of social emotional functioning. Youth who completedHall of Heroes significantly improved in their abilities to relate to others (both peers and family members) as well as to accept affection and express emotions with others, compared to youth who did not complete the SST intervention. Further, youth in the treatment condition showed a significantly greater decline in feelings of anxiety, depression, and hopelessness than did youth in the control condition. Both parents and youth reported high levels of engagement in and acceptability of theHall of Heroes. This study adds to the research literature, supporting the potential of a game-based SST platform for effectively helping youth develop prosocial social problem-solving skills.


---

## Body

## 1. Introduction

### 1.1. Social Relationships in Early Adolescence

The early adolescent period is a turbulent and stressful one for youth who must adapt and adjust to a myriad of concurrent life changes. At the same time as significant physical and cognitive growth is occurring, youth move from the smaller, more protective elementary school setting, with smaller classrooms and strong teacher support, to the larger, more impersonal middle school environment. In late elementary and middle school, academic demands increase considerably with more difficult assignments, higher expectations for independence, and more stringent grading [1, 2]. Concomitant with escalating academic demands, the social environment shifts from smaller, more protective settings to much larger, varied, and competitive social groups [3–6].The turbulence of these academic, social, and developmental shifts, combined with a decline in adult supervision, leads to a period of heightened vulnerability. For many youth, the early adolescent period is associated with loss of self-esteem, increased anxiety and depression, and drops in academic performance [3, 7]. And, adolescents with lower social skills (e.g., poor emotion regulation, difficulty with communication) and poorer peer relations are at greater risk of academic failure, behavioral problems, and emotional difficulties [8–11]. In contrast, positive social support and prosocial behaviors serve as protective factors, helping youth adjust more quickly to changing demands and fostering greater school engagement and academic success [2, 3, 8].


### 1.2. Social Skills Training

Social skills training (SST) interventions have repeatedly been shown to significantly improve social skills, behavior, and interpersonal relationships [12–15]. Effective SST can serve a protective function, as youth learn and practice those social skills that increase prosocial and/or inhibit maladaptive behaviors [16, 17]. For example, cooperation fosters companionship—a key function of peer relations—while impulsive and domineering behaviors undermine the development and maintenance of positive social relationships. As Peterson et al. [7] explain, “Improvements in social skills and problem-solving abilities allow the preadolescent to build a supportive network and engage in appropriate social activities that may ultimately contribute to reduced school drop-out rates as well as other indices of improved adaptive functioning” (pp. 289). Effective SST programs that improve social problem solving and social skills are essential for preparing youth for the transition to middle school [8, 10].Unfortunately, SST is traditionally delivered in person by trained clinicians or school staff which presents significant logistical barriers, including a lack of trained providers, scheduling difficulties, and significant time commitments, which too often prevent youth from participating in and benefiting from SST [14, 15]. Poor implementation fidelity by providers (e.g., nonadherence to the in-person treatment protocol) has also been shown to undermine outcomes for youth [7, 18]. And, lastly, SST is typically delivered in a group format with one provider to many youths. Research suggests tailoring interventions to the individual may be necessary to achieve intended gains in social and academic outcomes [19, 20].


### 1.3. Games for Impact

To help overcome the limitations of in-person SST, a growing number of intervention developers are employing emerging technologies [21]. Game-based platforms offer a cost-effective means to deliver program content in an accessible and engaging way, eliminate travel and provider staffing and training requirements, and standardize program delivery on a broad scale [19] (Merry et al., 2012). When digital games are dynamic and individualized, the user benefits from increased practice opportunities as well as a more tailored and personalized learning experience [19, 22–24]. Further, the engaging and interactive nature of games, and the relative anonymity of digital learning environments, means youth are often motivated to more fully and openly participate in game-based SST compared to traditional in-person SST [25, 26].There is growing support for the effectiveness of digital games for social, emotional, and behavioral development [21, 27–30]. Digital platforms have been shown to increase comprehension and recall of presented material over conventional instructional methods and produce greater generalization of acquired skills to real-life behavior (Clark et al., 2016) [31, 32]. Further, there is increasing evidence that digital environments are especially effective at targeting higher-order thinking skills of analysis, synthesis, and evaluation, which form the basis of social problem solving [28, 33].


### 1.4. The Development of Hall of Heroes

Drawing on the foundational principals of SST and digital game-based learning, 3C Institute developedHall of Heroes, an online single-player point-and-click adventure game wherein the youth navigate a virtual school-like world, engaging with other characters to solve tailored social problem-solving tasks. As illustrated in Figures 1 and 2, the fullHall of Heroes software includes a personalized avatar builder, a gameplay training scene, six baseline assessment scenes, seven tutorial scenes, a final skill mastery application scene, six cinematic cut scenes, and seven knowledge monitoring quizzes. Each of the first six scenes serves as an introduction to and baseline measure of one ofHall of Heroes’ six core social skills: impulse control, communication, cooperation, social initiation, empathy, and emotion regulation. Each of the seven tutorial scenes provides dynamic instructional content focusing on two or three particular social problem-solving strategies for these core skills (see Table 1 for scene overviews and learning objectives). The skill concepts covered in these scenes are reinforced in the knowledge assessments where players are asked a series of application questions relevant to the skill nuances explored in the preceding scenes and provided with dynamic feedback based on their answers.Table 1
Overview ofHall of Heroes six social emotional tutoring scenes with skill areas and learning objectives.
Scene Overview Skill Areas Learning Objectives The Replica Players work with two other students to put together a time machine replica during science class. Social InitiationCooperation Impulse Control Identify the correct group to initiate with and do so appropriatelyCooperate effectively, including negotiating about how to divide workUtilize available resources and organizational skills to effectively manage the assigned task  PowerFlip Players spend their free period attempting to join in with two different pairs of students playing a card game. Social InitiationEmotion Regulation Determine where to stand when joining inIdentify the appropriate person to talk to without interrupting Identify their own emotions after being rejectedChoose which group to re-join post rejection  Hero Simulation Players work with a group of fellow6th graders to produce a hero training video. CooperationCommunication Negotiate and compromise with another student to determine project rolesCoach other students in expressive nonverbal and verbal communication, focusing on tone of voice, facial expressions, and body language  Fire and Ice Players hone key interpersonal skills during their Hero-Civilians Relations class by playing three mini-games in a virtual reality simulator. Impulse ControlEmpathyEmotion Regulation Demonstrate impulse control in a response inhibition simulationAppropriately label dialog options based on their anticipated impact on another studentNavigate 3 “choose your own adventure” style activities which require stopping and thinking in highly emotional situations  The Escape Students have just witnessed the evil villain Dr. Klepto steal the teachers’ superpowers. Players must manage the fallout and rally other students to save the school. Emotion RegulationCommunicationEmpathy Regulate their own emotions with an activity (e.g., deep breathing and positive self-talk)Use receptive communication skills to determine the emotional states of other studentsHelp those students to regulate their emotions using appropriate activities  Surveillance Players watch security footage to figure out what is going on atHall of Heroes, and then work with other students to clarify details and form a theory and timeline of what happened. EmpathyImpulse Control Observe and interpret other characters’ emotions in contextBalance attention to detail and time constraints in a high-stakes situationDistinguish facts from assumptions  The Mission Players and other students work together to formulate a plan to travel back in time to stop Dr. Klepto and save the school. Impulse ControlCooperation Communication Work with other students to make and implement an action plan, which includes identifying the best team members for each stepCooperate with difficult others, using I statements and reflective listening to diffuse conflictFigure 1
Hall of Heroes scene flow diagram.Figure 2
Examples screenshots ofHall of Heroes game components.
(a)
Personalized avatar builder (b)
Scene selector (c)
Knowledge monitoring assessment (d)
Cinematic cut scenesHall of Heroes was developed by a multidisciplinary team comprising clinical and developmental psychologists, SST intervention developers, computer programmers, artists and animators, and experts in computer games for learning. The development process followed best practices in educational game design, with thoughtful consideration for the unique social and developmental challenges of early adolescence [23, 24, 28, 32, 34]. These considerations led us to employ multiple gameplay styles, varying the settings and backgrounds for each scene, and to incorporate a compelling storyline through both playable scenes and cinematic cut scenes, resulting in a motivating and engaging game environment where all types of players can find something to attract them [26, 35–37]. Figure 3 presents a sample of the range of gameplay experiences inHall of Heroes, which include simulations, drill and practice mini-games, opportunities for open exploration, dialog driven content, and action-based challenges. By incorporating direct instruction, varied practice exercises, dynamic feedback, and self-directed elements, the game is able to effectively target multiple modes of learning [23, 26, 34, 37].Figure 3
Example screenshots ofHall of Heroes gameplay.
(a)
Simulation: players act as the director during a video shoot (b)
Dialog menu: players discuss where to sit during lunch (c)
Skills application: players demonstrate receptive communication in a notes-taking exercise (d)
Mini-game: players practice deep breathing techniques (e)
Practice exercise: players practice using a combination lock to open their locker (f)
Practice exercise: players choose where to stand when joining inSituating the game in a school-like setting allows for the inclusion of tasks that directly address the routine aspects of middle school, such as changing classes and using a locker, which repeatedly come up as areas of pretransition concern and anxiety for youth [7, 38]. In addition, the school setting allowed us to simulate challenging social situations unique to a school environment, such as choosing a table for lunch and understanding when and how to engage a teacher for help in a bullying situation. Incorporating these common transition experiences into a superhero world serves to balance fantasy and realism, maintaining engagement and allowing for deeper learning and skills transfer [32].To further develop this rich narrative world,Hall of Heroes is populated with a large and diverse cast of nonplayable characters (NPCs) which serve narrative and pedagogical roles that mirror traditional instructional roles: peer, teacher, mentor, learning companion, etc. Experts in the field of educational game design agree that the inclusion of interactive embodied social pedagogical agents enhances student motivation and engagement and improves learning outcomes including retention and skills transfer (e.g., [39–41]). NPCs as embodied pedagogical agents support the social cognitive aspects of learning, serving as an interactive bridge to help players understand complex concepts [39, 42]. Rather than simply presenting information, the NPCs inHall of Heroes engage and motivate players through direct experiences with the game world to actively participate in the construction of social problem-solving knowledge, for example, by guiding players to look for clues in school security footage in order to check their assumptions about a situation. NPCs also serve to model and guide players through difficult cognitive processes, for example, by facilitating positive self-talk through conversation and branching dialog menus.In the current study, we conducted an initial investigation of the efficacy and acceptability of theHall of Heroes game for SST using a randomized-control design. We expected youth who interacted withHall of Heroes would demonstrate significantly greater improvements in social, emotional, and behavioral outcomes compared to youth in the waitlist control condition who did not complete the game.


## 1.1. Social Relationships in Early Adolescence

The early adolescent period is a turbulent and stressful one for youth who must adapt and adjust to a myriad of concurrent life changes. At the same time as significant physical and cognitive growth is occurring, youth move from the smaller, more protective elementary school setting, with smaller classrooms and strong teacher support, to the larger, more impersonal middle school environment. In late elementary and middle school, academic demands increase considerably with more difficult assignments, higher expectations for independence, and more stringent grading [1, 2]. Concomitant with escalating academic demands, the social environment shifts from smaller, more protective settings to much larger, varied, and competitive social groups [3–6].The turbulence of these academic, social, and developmental shifts, combined with a decline in adult supervision, leads to a period of heightened vulnerability. For many youth, the early adolescent period is associated with loss of self-esteem, increased anxiety and depression, and drops in academic performance [3, 7]. And, adolescents with lower social skills (e.g., poor emotion regulation, difficulty with communication) and poorer peer relations are at greater risk of academic failure, behavioral problems, and emotional difficulties [8–11]. In contrast, positive social support and prosocial behaviors serve as protective factors, helping youth adjust more quickly to changing demands and fostering greater school engagement and academic success [2, 3, 8].


## 1.2. Social Skills Training

Social skills training (SST) interventions have repeatedly been shown to significantly improve social skills, behavior, and interpersonal relationships [12–15]. Effective SST can serve a protective function, as youth learn and practice those social skills that increase prosocial and/or inhibit maladaptive behaviors [16, 17]. For example, cooperation fosters companionship—a key function of peer relations—while impulsive and domineering behaviors undermine the development and maintenance of positive social relationships. As Peterson et al. [7] explain, “Improvements in social skills and problem-solving abilities allow the preadolescent to build a supportive network and engage in appropriate social activities that may ultimately contribute to reduced school drop-out rates as well as other indices of improved adaptive functioning” (pp. 289). Effective SST programs that improve social problem solving and social skills are essential for preparing youth for the transition to middle school [8, 10].Unfortunately, SST is traditionally delivered in person by trained clinicians or school staff which presents significant logistical barriers, including a lack of trained providers, scheduling difficulties, and significant time commitments, which too often prevent youth from participating in and benefiting from SST [14, 15]. Poor implementation fidelity by providers (e.g., nonadherence to the in-person treatment protocol) has also been shown to undermine outcomes for youth [7, 18]. And, lastly, SST is typically delivered in a group format with one provider to many youths. Research suggests tailoring interventions to the individual may be necessary to achieve intended gains in social and academic outcomes [19, 20].


## 1.3. Games for Impact

To help overcome the limitations of in-person SST, a growing number of intervention developers are employing emerging technologies [21]. Game-based platforms offer a cost-effective means to deliver program content in an accessible and engaging way, eliminate travel and provider staffing and training requirements, and standardize program delivery on a broad scale [19] (Merry et al., 2012). When digital games are dynamic and individualized, the user benefits from increased practice opportunities as well as a more tailored and personalized learning experience [19, 22–24]. Further, the engaging and interactive nature of games, and the relative anonymity of digital learning environments, means youth are often motivated to more fully and openly participate in game-based SST compared to traditional in-person SST [25, 26].There is growing support for the effectiveness of digital games for social, emotional, and behavioral development [21, 27–30]. Digital platforms have been shown to increase comprehension and recall of presented material over conventional instructional methods and produce greater generalization of acquired skills to real-life behavior (Clark et al., 2016) [31, 32]. Further, there is increasing evidence that digital environments are especially effective at targeting higher-order thinking skills of analysis, synthesis, and evaluation, which form the basis of social problem solving [28, 33].


## 1.4. The Development of Hall of Heroes

Drawing on the foundational principals of SST and digital game-based learning, 3C Institute developedHall of Heroes, an online single-player point-and-click adventure game wherein the youth navigate a virtual school-like world, engaging with other characters to solve tailored social problem-solving tasks. As illustrated in Figures 1 and 2, the fullHall of Heroes software includes a personalized avatar builder, a gameplay training scene, six baseline assessment scenes, seven tutorial scenes, a final skill mastery application scene, six cinematic cut scenes, and seven knowledge monitoring quizzes. Each of the first six scenes serves as an introduction to and baseline measure of one ofHall of Heroes’ six core social skills: impulse control, communication, cooperation, social initiation, empathy, and emotion regulation. Each of the seven tutorial scenes provides dynamic instructional content focusing on two or three particular social problem-solving strategies for these core skills (see Table 1 for scene overviews and learning objectives). The skill concepts covered in these scenes are reinforced in the knowledge assessments where players are asked a series of application questions relevant to the skill nuances explored in the preceding scenes and provided with dynamic feedback based on their answers.Table 1
Overview ofHall of Heroes six social emotional tutoring scenes with skill areas and learning objectives.
Scene Overview Skill Areas Learning Objectives The Replica Players work with two other students to put together a time machine replica during science class. Social InitiationCooperation Impulse Control Identify the correct group to initiate with and do so appropriatelyCooperate effectively, including negotiating about how to divide workUtilize available resources and organizational skills to effectively manage the assigned task  PowerFlip Players spend their free period attempting to join in with two different pairs of students playing a card game. Social InitiationEmotion Regulation Determine where to stand when joining inIdentify the appropriate person to talk to without interrupting Identify their own emotions after being rejectedChoose which group to re-join post rejection  Hero Simulation Players work with a group of fellow6th graders to produce a hero training video. CooperationCommunication Negotiate and compromise with another student to determine project rolesCoach other students in expressive nonverbal and verbal communication, focusing on tone of voice, facial expressions, and body language  Fire and Ice Players hone key interpersonal skills during their Hero-Civilians Relations class by playing three mini-games in a virtual reality simulator. Impulse ControlEmpathyEmotion Regulation Demonstrate impulse control in a response inhibition simulationAppropriately label dialog options based on their anticipated impact on another studentNavigate 3 “choose your own adventure” style activities which require stopping and thinking in highly emotional situations  The Escape Students have just witnessed the evil villain Dr. Klepto steal the teachers’ superpowers. Players must manage the fallout and rally other students to save the school. Emotion RegulationCommunicationEmpathy Regulate their own emotions with an activity (e.g., deep breathing and positive self-talk)Use receptive communication skills to determine the emotional states of other studentsHelp those students to regulate their emotions using appropriate activities  Surveillance Players watch security footage to figure out what is going on atHall of Heroes, and then work with other students to clarify details and form a theory and timeline of what happened. EmpathyImpulse Control Observe and interpret other characters’ emotions in contextBalance attention to detail and time constraints in a high-stakes situationDistinguish facts from assumptions  The Mission Players and other students work together to formulate a plan to travel back in time to stop Dr. Klepto and save the school. Impulse ControlCooperation Communication Work with other students to make and implement an action plan, which includes identifying the best team members for each stepCooperate with difficult others, using I statements and reflective listening to diffuse conflictFigure 1
Hall of Heroes scene flow diagram.Figure 2
Examples screenshots ofHall of Heroes game components.
(a)
Personalized avatar builder (b)
Scene selector (c)
Knowledge monitoring assessment (d)
Cinematic cut scenesHall of Heroes was developed by a multidisciplinary team comprising clinical and developmental psychologists, SST intervention developers, computer programmers, artists and animators, and experts in computer games for learning. The development process followed best practices in educational game design, with thoughtful consideration for the unique social and developmental challenges of early adolescence [23, 24, 28, 32, 34]. These considerations led us to employ multiple gameplay styles, varying the settings and backgrounds for each scene, and to incorporate a compelling storyline through both playable scenes and cinematic cut scenes, resulting in a motivating and engaging game environment where all types of players can find something to attract them [26, 35–37]. Figure 3 presents a sample of the range of gameplay experiences inHall of Heroes, which include simulations, drill and practice mini-games, opportunities for open exploration, dialog driven content, and action-based challenges. By incorporating direct instruction, varied practice exercises, dynamic feedback, and self-directed elements, the game is able to effectively target multiple modes of learning [23, 26, 34, 37].Figure 3
Example screenshots ofHall of Heroes gameplay.
(a)
Simulation: players act as the director during a video shoot (b)
Dialog menu: players discuss where to sit during lunch (c)
Skills application: players demonstrate receptive communication in a notes-taking exercise (d)
Mini-game: players practice deep breathing techniques (e)
Practice exercise: players practice using a combination lock to open their locker (f)
Practice exercise: players choose where to stand when joining inSituating the game in a school-like setting allows for the inclusion of tasks that directly address the routine aspects of middle school, such as changing classes and using a locker, which repeatedly come up as areas of pretransition concern and anxiety for youth [7, 38]. In addition, the school setting allowed us to simulate challenging social situations unique to a school environment, such as choosing a table for lunch and understanding when and how to engage a teacher for help in a bullying situation. Incorporating these common transition experiences into a superhero world serves to balance fantasy and realism, maintaining engagement and allowing for deeper learning and skills transfer [32].To further develop this rich narrative world,Hall of Heroes is populated with a large and diverse cast of nonplayable characters (NPCs) which serve narrative and pedagogical roles that mirror traditional instructional roles: peer, teacher, mentor, learning companion, etc. Experts in the field of educational game design agree that the inclusion of interactive embodied social pedagogical agents enhances student motivation and engagement and improves learning outcomes including retention and skills transfer (e.g., [39–41]). NPCs as embodied pedagogical agents support the social cognitive aspects of learning, serving as an interactive bridge to help players understand complex concepts [39, 42]. Rather than simply presenting information, the NPCs inHall of Heroes engage and motivate players through direct experiences with the game world to actively participate in the construction of social problem-solving knowledge, for example, by guiding players to look for clues in school security footage in order to check their assumptions about a situation. NPCs also serve to model and guide players through difficult cognitive processes, for example, by facilitating positive self-talk through conversation and branching dialog menus.In the current study, we conducted an initial investigation of the efficacy and acceptability of theHall of Heroes game for SST using a randomized-control design. We expected youth who interacted withHall of Heroes would demonstrate significantly greater improvements in social, emotional, and behavioral outcomes compared to youth in the waitlist control condition who did not complete the game.


## 2. Methods

### 2.1. Participants

Participants were recruited via postings on national parenting, game research, and school listservs and social media sites. Interested parents completed an online survey to determine eligibility. To be eligible to participate, youth must have (1) been attending the 5th or 6th grades; (2) received instruction in a regular education classroom for at least 40% of the school day; (3) been English language proficient; and (4) had access to a computer with Internet use at home.Eligible participants (N = 32) were randomly assigned to either the treatment (TX) or waitlist control (WLC). Nine percent of participants were attrited over the course of the study (i.e., failed to complete post-data collection tasks), resulting in a final sample of 29 youths. Chi-squared tests revealed no significant differences between participants who completed the study and those who did not based on age, race, gender, or study condition. The final sample of youth ranged in age from nine to thirteen years (M = 11.35, SD = 0.88), with 76% male and 31% representing a racial or ethnic minority group. There were no significant demographic differences across the two conditions.


### 2.2. Measures

Positive Behavioral and Emotional Skills. Youth behavioral and emotional strengths were measured usingThe Behavioral and Emotional Rating Scale, Second Edition (BERS-2 [43]), a 58-item strength-based assessment that measures youth positive emotions, behaviors, and interpersonal skills. Parents reported how many statements, such as “acknowledges painful feelings,” “considers consequences of own behavior,” and “completes homework regularly,” were like their child over the past 30 days on a 4-point scale ranging from 0 (not at all) to 3 (very much). The BERS-2 compositeStrength Index was calculated by summing across all items and converting this raw score to a standard score (M=100, SD=15). Further, the composite score was broken down into its five subscales:interpersonal strength(strengths in how the child relates to others), affective strength(strengths in how the accepts affection and expresses emotions),family involvement(strengths in the child’s relationships with family members), intrapersonal strength(child’s internal emotional strengths, outlook on one’s own competencies and accomplishments), andschool functioning(strengths in completing school-related tasks). Raw scores for each subscale were computed by summing scores for all relevant items and converting them to standard scores (M=10, SD=3). Higher scores reflect greater behavioral and emotional strength in that area. The BERS-2Strength Index demonstrated good reliability (Cronbach’s α = .97) and prior factor analyses have established the measure’s content, criterion, and construct validity [43].Psychosocial Distress. Youth psychosocial distress was measured using the 64-item parent-reportYouth Outcomes Questionnaire, Second Edition (Y-OQ [44]). Parents were asked to think about their child over the past 7 days and report how true statements were for their child on a 5-point scale from 0 (never or almost never) to 4 (almost always or always). The Y-OQTotal Distressscore was computed by summing across all items and respective items were combined for six subscales:intrapersonal distress(18 items covering anxiety, depression, and hopelessness), somatic(8 items including headaches, dizziness, and stomachaches), interpersonal relations(10 items covering arguing, defiance, and communication problems), social problems (8 delinquent or aggressive behavior items),behavioral dysfunction(11 items covering organization, concentration, handling frustration, and ADHD-related symptoms), andcritical items (9 items consisting of symptoms often found in youth receiving inpatient services, such as paranoid ideation, hallucinations, mania, and suicidal feelings). Higher scores on each scale indicate higher levels of psychosocial distress. The Y-OQ showed high reliability (Cronbach’s α = .97) and its publishers detail several studies establishing content, criterion, and construct validity with both clinical and nonclinical (community) samples [44].Game Evaluation. Following completion, youth and parents were asked to provide feedback on theHall of Heroes program via evaluation surveys. Youth were asked to rate how much they agreed with statements regarding their engagement and liking ofHall of Heroes. Parents rated the extent to which they agreed with statements such as “I would recommendHall of Heroes to other parents of 5th or 6th grade youth” and “Hall of Heroes was easy for my child to understand.” Youth and parent evaluations were made using a 5-point scale with 1 = strongly disagree and 5 = strongly agree. Open-ended responses were also collected, to collect more detailed feedback on youth and parent impressions of the SEL intervention.


### 2.3. Procedures

Precautions were taken to ensure study ethics and protection of human subjects. The study protocol was approved by 3C Institute’s institutional review board. Parent consent and youth assent were obtained from all participants prior to participation. All parts of the study were completed online through a secure project website.All parents in the TX and WLC groups completed the BERS-2 and Y-OQ at pretest. The week following pretest data collection, youth in the TX group began interacting withHall of Heroes. The game was released in four segments, with one segment released per week for four weeks; youth had one week to complete each segment. Within one week following the TX group’s four-week interaction with the program, parents in both the TX and WLC groups completed the same set of outcome measures again, as posttest measures, and youth and parents in the TX group completed product evaluation surveys.


## 2.1. Participants

Participants were recruited via postings on national parenting, game research, and school listservs and social media sites. Interested parents completed an online survey to determine eligibility. To be eligible to participate, youth must have (1) been attending the 5th or 6th grades; (2) received instruction in a regular education classroom for at least 40% of the school day; (3) been English language proficient; and (4) had access to a computer with Internet use at home.Eligible participants (N = 32) were randomly assigned to either the treatment (TX) or waitlist control (WLC). Nine percent of participants were attrited over the course of the study (i.e., failed to complete post-data collection tasks), resulting in a final sample of 29 youths. Chi-squared tests revealed no significant differences between participants who completed the study and those who did not based on age, race, gender, or study condition. The final sample of youth ranged in age from nine to thirteen years (M = 11.35, SD = 0.88), with 76% male and 31% representing a racial or ethnic minority group. There were no significant demographic differences across the two conditions.


## 2.2. Measures

Positive Behavioral and Emotional Skills. Youth behavioral and emotional strengths were measured usingThe Behavioral and Emotional Rating Scale, Second Edition (BERS-2 [43]), a 58-item strength-based assessment that measures youth positive emotions, behaviors, and interpersonal skills. Parents reported how many statements, such as “acknowledges painful feelings,” “considers consequences of own behavior,” and “completes homework regularly,” were like their child over the past 30 days on a 4-point scale ranging from 0 (not at all) to 3 (very much). The BERS-2 compositeStrength Index was calculated by summing across all items and converting this raw score to a standard score (M=100, SD=15). Further, the composite score was broken down into its five subscales:interpersonal strength(strengths in how the child relates to others), affective strength(strengths in how the accepts affection and expresses emotions),family involvement(strengths in the child’s relationships with family members), intrapersonal strength(child’s internal emotional strengths, outlook on one’s own competencies and accomplishments), andschool functioning(strengths in completing school-related tasks). Raw scores for each subscale were computed by summing scores for all relevant items and converting them to standard scores (M=10, SD=3). Higher scores reflect greater behavioral and emotional strength in that area. The BERS-2Strength Index demonstrated good reliability (Cronbach’s α = .97) and prior factor analyses have established the measure’s content, criterion, and construct validity [43].Psychosocial Distress. Youth psychosocial distress was measured using the 64-item parent-reportYouth Outcomes Questionnaire, Second Edition (Y-OQ [44]). Parents were asked to think about their child over the past 7 days and report how true statements were for their child on a 5-point scale from 0 (never or almost never) to 4 (almost always or always). The Y-OQTotal Distressscore was computed by summing across all items and respective items were combined for six subscales:intrapersonal distress(18 items covering anxiety, depression, and hopelessness), somatic(8 items including headaches, dizziness, and stomachaches), interpersonal relations(10 items covering arguing, defiance, and communication problems), social problems (8 delinquent or aggressive behavior items),behavioral dysfunction(11 items covering organization, concentration, handling frustration, and ADHD-related symptoms), andcritical items (9 items consisting of symptoms often found in youth receiving inpatient services, such as paranoid ideation, hallucinations, mania, and suicidal feelings). Higher scores on each scale indicate higher levels of psychosocial distress. The Y-OQ showed high reliability (Cronbach’s α = .97) and its publishers detail several studies establishing content, criterion, and construct validity with both clinical and nonclinical (community) samples [44].Game Evaluation. Following completion, youth and parents were asked to provide feedback on theHall of Heroes program via evaluation surveys. Youth were asked to rate how much they agreed with statements regarding their engagement and liking ofHall of Heroes. Parents rated the extent to which they agreed with statements such as “I would recommendHall of Heroes to other parents of 5th or 6th grade youth” and “Hall of Heroes was easy for my child to understand.” Youth and parent evaluations were made using a 5-point scale with 1 = strongly disagree and 5 = strongly agree. Open-ended responses were also collected, to collect more detailed feedback on youth and parent impressions of the SEL intervention.


## 2.3. Procedures

Precautions were taken to ensure study ethics and protection of human subjects. The study protocol was approved by 3C Institute’s institutional review board. Parent consent and youth assent were obtained from all participants prior to participation. All parts of the study were completed online through a secure project website.All parents in the TX and WLC groups completed the BERS-2 and Y-OQ at pretest. The week following pretest data collection, youth in the TX group began interacting withHall of Heroes. The game was released in four segments, with one segment released per week for four weeks; youth had one week to complete each segment. Within one week following the TX group’s four-week interaction with the program, parents in both the TX and WLC groups completed the same set of outcome measures again, as posttest measures, and youth and parents in the TX group completed product evaluation surveys.


## 3. Results and Discussion

Preliminary analyses were conducted using a Multivariate Analysis of Variance (MANOVA) with condition as the between-subjects factor to determine whether scores on the BERS-2 and the Y-OQ differed at baseline (preintervention) across the two conditions. These analyses revealed no significant difference between groups at baseline on any subscale of the BERS-2 or the Y-OQ. However, given the small sample size, we elected to use an Analysis of Covariance (ANCOVA) approach in order to control for baseline scores when analyzing intervention effects [45, 46].


### 3.1. Positive Behavioral and Emotional Skills

As can be seen in Table2, results of the ANCOVA revealed significant improvements in positive behavioral and emotional skills for youth in the treatment condition, as measured by the BERS-2. As expected, parents reported that youth who completedHall of Heroes showed significantly greater growth in their totalStrength Index (a measure of youth overall behavioral and emotional strengths) compared to WLC youth who did not complete the intervention. Further, when this index was broken down into its composite subscales, youth in the treatment condition showed particular gains in three areas:  interpersonal strength, affective strength, and family involvement. Thus, youth who completedHall of Heroes significantly improved in their abilities to relate to others (both peers and family members) as well as to accept affection and express emotions with others, whereas youth who did not completeHall of Heroes showed essentially no change in these areas. These results were expected given thatHall of Heroes directly targets social skills that are critical for positive interpersonal relationships, including communication, cooperation, and social initiation as well as emotional awareness and emotion regulation skills which would directly impact the youth’s affective abilities.Table 2
Summary of relevant descriptive information, ANCOVA statistics, and effect sizes for parent-reported outcomes on the BERS-2.
Mean (SE) ANCOVA Pre Post F p-value Effect Size (ηp2) TotalStrength Index Treatment 92.73 (3.73) 110.33 (4.39) 5.96 .022 .19 Wait-list Control 94.50 (6.01) 95.43 (6.21) Interpersonal Strength Treatment 6.73 (.67) 9.07 (.67) 19.25 <.001 .43 Wait-list Control 7.07 (.74) 6.43 (.75) Affective Strength Treatment 8.13 (.77) 10.20 (.73) 6.29 .019 .20 Wait-list Control 7.43 (.80) 7.43 (.98) Family Involvement Treatment 7.07 (.69) 9.60 (.77) 14.32 <.001 .36 Wait-list Control 7.14 (.58) 7.29 (.62) Intrapersonal Strength Treatment 6.40 (.47) 9.20 (.84) 2.67 .115 .09 Wait-list Control 7.00 (.89) 7.57 (.83) School Functioning Treatment 7.80 (.55) 9.60 (.62) .91 .349 .03 Wait-list Control 8.57 (1.07) 8.64 (1.01)No significant difference by treatment condition was found for the BERS-2intrapersonal strengthsubscale.It may be that the duration of the study was too brief (4 weeks) to impact youth’s internal emotional strengths and opinions of their own competencies and accomplishments, which may take more time and real-world experience to alter than possible with this study design. No significant difference was found for the BERS-2school functioningsubscale either. However,Hall of Heroes does not directly address academics or school-related tasks, so lack of a treatment effect for this subscale may be expected.While intrapersonal and school academics were not directly targeted by theHall of Heroes intervention, improvements for these subscales may emerge with greater exposure to the intervention and/or more time for these more distal outcomes to be impacted by social skill improvements for the youth. As the youth practice and implement the social skills taught inHall of Heroes, effects on self-efficacy and academic competence may emerge. Future research with a longer intervention period could directly test this hypothesis.


### 3.2. Psychosocial Distress

As can be seen in Table3, ANCOVA results demonstrated TX youth experienced significantly lowerOverall Psychosocial Distress on the Y-OQ at postintervention compared to WLC youth, after controlling for preintervention scores. This result suggests thatHall of Heroes’ bolstering of social emotional skills may be key to helping youth cope with stressful situations. Results further indicated that this overall significant effect was largely due to significant improvement inintrapersonal distress for youth in the TX condition. Thus, youth who completedHall of Heroes showed a significantly greater decline in feelings of anxiety, depression, and hopelessness than did youth who did not participate in the intervention.Table 3
Summary of relevant descriptive information, ANCOVA statistics, and effect sizes for parent-reported outcomes on the Y-OQ.
Mean (SE) ANCOVA Pre Post F p-value Effect Size (ηp2) Overall Distress Treatment 82.07 (6.46) 54.20 (7.06) 4.57 .042 .15 Wait-list Control 73.79 (9.76) 65.57 (8.89) Intrapersonal Distress Treatment 29.47 (2.27) 18.73 (2.72) 8.23 .008 .24 Wait-list Control 24.29 (2.75) 21.71 (2.81) Somatic Treatment 9.73 (1.35) 6.07 (1.34) 3.58 .070 .12 Wait-list Control 8.29 (1.58) 7.79 (1.52) Critical Items Treatment 9.53 (1.75) 5.33 (.92) .86 .103 .10 Wait-list Control 8.43 (1.92) 6.64 (1.30) Interpersonal Relationships Treatment 8.07 (1.36) 5.00 (1.83) .99 .329 .04 Wait-list Control 9.43 (1.61) 7.93 (1.62) Social Problems Treatment 3.80 (1.23) 2.60 (1.20) .98 .332 .04 Wait-list Control 5.14 (1.42) 4.71 (1.49) Behavioral Dysfunction Treatment 21.47 (1.93) 16.47 (1.54) .24 .632 .01 Wait-list Control 18.21 (2.19) 16.79 (1.73)While not statistically significant, two other Y-OQ subscales showed trends suggesting youth who completedHall of Heroes appeared to experience fewer symptoms of somatic distress (e.g., headache, stomach, and dizziness) and fewer critical psychological symptoms (e.g., thoughts of suicide, paranoia, and mania) compared to youth who did not complete the intervention. We know, from the literature, youth who experience social problems often develop somatic complaints, either as a function of the anxiety associated with interpersonal problems or as a coping mechanism to avoid socially stressful situations, such as school [9]. The trend towards fewer somatic symptoms may be an indirect effect from the overall lessening of psychosocial distress in the area of anxiety for youth in the treatment condition. Similarly, lower psychosocial distress in the areas of depression and hopelessness would be expected to decrease the level of serious psychological (critical) symptoms experienced by a youth.No statistically significant between-group differences were found on the Y-OQ for the interpersonal relationships or social problems subscales. This finding is somewhat surprising given the significant findings on similar interpersonal strength subscale on the BERS-2, reported above. However, for the Y-OQ,interpersonal relationsfocuses on arguing, defiance, and negative communication problems andsocial problems focuses on delinquent and aggressive behavior problems. While greater (but nonsignificant) declines in these problem areas were seen for youth in the TX condition compared to those in the WLC condition, these areas represent antisocial behavior problems that are not the primary focus ofHall of Heroes.Hall of Heroes focuses on teaching prosocial skills and behaviors and positive social problem solving. We know that antisocial behavior problems are more resistant to change [13] and that positive impact on antisocial behavior problems may take longer than the 4-week intervention period of this study. It may be that declines in these antisocial behavior problems would emerge with greater exposure toHall of Heroes as well as greater real-world experience using prosocial skills that are antithetical to antisocial behaviors.With regard to thebehavioral dysfunctionof the Y-OQ, this subscale focuses on ADHD-related symptoms, such as disorganization, poor concentration, and trouble handling frustration, which are not directly addressed by theHall of Heroes intervention. Thus, it is not too surprising that there was no significant group difference for this particular subscale. However, whileHall of Heroes is not designed as a treatment tool for any specific disorder (e.g., Attention Deficit Hyperactivity Disorder, Conduct Disorder), it is possible that improvements in these types of behavioral problem areas may arise over time as youth practice new social emotional skills in the real-world. Future longitudinal research would be needed to test whether group differences in antisocial and ADHD-related behavior problems emerge over time for youth.


### 3.3. Hall of Heroes Evaluation

BecauseHall of Heroes was designed to be used independently by youth and because greater engagement in intervention is associated with greater skill gains (Clark et al., 2016) [47], it was important to examine how youth evaluated the game. In general, as can be seen in Table 4, both youth and parents ratedHall of Heroes favorably across all areas. Parents supportedHall of Heroes as engaging, and youth reported they found the game fun and enjoyed both the characters and story. Also of note were youths’ high marks for interest in playing more games likeHall of Heroes and parents’ belief thatHall of Heroes is an effective tool for teaching social skills with youth.Table 4
Means (SD) of youth and parent evaluations ofHall of Heroes.
Youth Ratings Mean (SD) Overall, I likedHall of Heroes. 4.23 (.83) I would like to play more games likeHall of Heroes. 4.38 (.65) I thoughtHall of Heroes was fun to play. 4.31 (.85) I liked the characters inHall of Heroes. 4.23 (.92) I liked the pictures/graphics inHall of Heroes. 3.92 (1.38) I liked the story inHall of Heroes. 4.15 (1.14)  Parent Ratings Mean (SD)  Hall of Heroes is of high overall quality. 4.00 (.65) Hall of Heroes was fun and engaging for my child. 4.27 (.80) Hall of Heroes was easy for my child to understand. 4.00 (.85) My child’s overall experience withHall of Heroes was positive. 4.13 (.83) Hall of Heroes is an effective tool for teaching students social skills. 4.20 (.68) Hall of Heroes is an effective tool for preparing students for middle school. 4.27 (.59) I would recommendHall of Heroes to other parents of 5th or 6th grade students. 4.33 (.82) Note. All ratings were made on a 5-point scale from 1 (strongly disagree) to 5 (strongly agree).Follow-up requests for open-ended feedback helped to put these ratings into context. When the youth were asked what they learned inHall of Heroes, nearly all noted at least one of the program goals, including “cooperation with others,” “respect for others,” “how to deal with feelings,” and “communication and working together,” to name a few. Representative comments from parents were like “my child learned about working together for a common goal,” “my child had fun enough to keep her wanting to play, yet helped her with some new ideas about how to interact with others,” and “there were more than few scenarios that my child encountered that forced him to reevaluate and change directions,” among other comments. Further, parents reported they were likely to recommendHall of Heroes to other parents of adolescents.


## 3.1. Positive Behavioral and Emotional Skills

As can be seen in Table2, results of the ANCOVA revealed significant improvements in positive behavioral and emotional skills for youth in the treatment condition, as measured by the BERS-2. As expected, parents reported that youth who completedHall of Heroes showed significantly greater growth in their totalStrength Index (a measure of youth overall behavioral and emotional strengths) compared to WLC youth who did not complete the intervention. Further, when this index was broken down into its composite subscales, youth in the treatment condition showed particular gains in three areas:  interpersonal strength, affective strength, and family involvement. Thus, youth who completedHall of Heroes significantly improved in their abilities to relate to others (both peers and family members) as well as to accept affection and express emotions with others, whereas youth who did not completeHall of Heroes showed essentially no change in these areas. These results were expected given thatHall of Heroes directly targets social skills that are critical for positive interpersonal relationships, including communication, cooperation, and social initiation as well as emotional awareness and emotion regulation skills which would directly impact the youth’s affective abilities.Table 2
Summary of relevant descriptive information, ANCOVA statistics, and effect sizes for parent-reported outcomes on the BERS-2.
Mean (SE) ANCOVA Pre Post F p-value Effect Size (ηp2) TotalStrength Index Treatment 92.73 (3.73) 110.33 (4.39) 5.96 .022 .19 Wait-list Control 94.50 (6.01) 95.43 (6.21) Interpersonal Strength Treatment 6.73 (.67) 9.07 (.67) 19.25 <.001 .43 Wait-list Control 7.07 (.74) 6.43 (.75) Affective Strength Treatment 8.13 (.77) 10.20 (.73) 6.29 .019 .20 Wait-list Control 7.43 (.80) 7.43 (.98) Family Involvement Treatment 7.07 (.69) 9.60 (.77) 14.32 <.001 .36 Wait-list Control 7.14 (.58) 7.29 (.62) Intrapersonal Strength Treatment 6.40 (.47) 9.20 (.84) 2.67 .115 .09 Wait-list Control 7.00 (.89) 7.57 (.83) School Functioning Treatment 7.80 (.55) 9.60 (.62) .91 .349 .03 Wait-list Control 8.57 (1.07) 8.64 (1.01)No significant difference by treatment condition was found for the BERS-2intrapersonal strengthsubscale.It may be that the duration of the study was too brief (4 weeks) to impact youth’s internal emotional strengths and opinions of their own competencies and accomplishments, which may take more time and real-world experience to alter than possible with this study design. No significant difference was found for the BERS-2school functioningsubscale either. However,Hall of Heroes does not directly address academics or school-related tasks, so lack of a treatment effect for this subscale may be expected.While intrapersonal and school academics were not directly targeted by theHall of Heroes intervention, improvements for these subscales may emerge with greater exposure to the intervention and/or more time for these more distal outcomes to be impacted by social skill improvements for the youth. As the youth practice and implement the social skills taught inHall of Heroes, effects on self-efficacy and academic competence may emerge. Future research with a longer intervention period could directly test this hypothesis.


## 3.2. Psychosocial Distress

As can be seen in Table3, ANCOVA results demonstrated TX youth experienced significantly lowerOverall Psychosocial Distress on the Y-OQ at postintervention compared to WLC youth, after controlling for preintervention scores. This result suggests thatHall of Heroes’ bolstering of social emotional skills may be key to helping youth cope with stressful situations. Results further indicated that this overall significant effect was largely due to significant improvement inintrapersonal distress for youth in the TX condition. Thus, youth who completedHall of Heroes showed a significantly greater decline in feelings of anxiety, depression, and hopelessness than did youth who did not participate in the intervention.Table 3
Summary of relevant descriptive information, ANCOVA statistics, and effect sizes for parent-reported outcomes on the Y-OQ.
Mean (SE) ANCOVA Pre Post F p-value Effect Size (ηp2) Overall Distress Treatment 82.07 (6.46) 54.20 (7.06) 4.57 .042 .15 Wait-list Control 73.79 (9.76) 65.57 (8.89) Intrapersonal Distress Treatment 29.47 (2.27) 18.73 (2.72) 8.23 .008 .24 Wait-list Control 24.29 (2.75) 21.71 (2.81) Somatic Treatment 9.73 (1.35) 6.07 (1.34) 3.58 .070 .12 Wait-list Control 8.29 (1.58) 7.79 (1.52) Critical Items Treatment 9.53 (1.75) 5.33 (.92) .86 .103 .10 Wait-list Control 8.43 (1.92) 6.64 (1.30) Interpersonal Relationships Treatment 8.07 (1.36) 5.00 (1.83) .99 .329 .04 Wait-list Control 9.43 (1.61) 7.93 (1.62) Social Problems Treatment 3.80 (1.23) 2.60 (1.20) .98 .332 .04 Wait-list Control 5.14 (1.42) 4.71 (1.49) Behavioral Dysfunction Treatment 21.47 (1.93) 16.47 (1.54) .24 .632 .01 Wait-list Control 18.21 (2.19) 16.79 (1.73)While not statistically significant, two other Y-OQ subscales showed trends suggesting youth who completedHall of Heroes appeared to experience fewer symptoms of somatic distress (e.g., headache, stomach, and dizziness) and fewer critical psychological symptoms (e.g., thoughts of suicide, paranoia, and mania) compared to youth who did not complete the intervention. We know, from the literature, youth who experience social problems often develop somatic complaints, either as a function of the anxiety associated with interpersonal problems or as a coping mechanism to avoid socially stressful situations, such as school [9]. The trend towards fewer somatic symptoms may be an indirect effect from the overall lessening of psychosocial distress in the area of anxiety for youth in the treatment condition. Similarly, lower psychosocial distress in the areas of depression and hopelessness would be expected to decrease the level of serious psychological (critical) symptoms experienced by a youth.No statistically significant between-group differences were found on the Y-OQ for the interpersonal relationships or social problems subscales. This finding is somewhat surprising given the significant findings on similar interpersonal strength subscale on the BERS-2, reported above. However, for the Y-OQ,interpersonal relationsfocuses on arguing, defiance, and negative communication problems andsocial problems focuses on delinquent and aggressive behavior problems. While greater (but nonsignificant) declines in these problem areas were seen for youth in the TX condition compared to those in the WLC condition, these areas represent antisocial behavior problems that are not the primary focus ofHall of Heroes.Hall of Heroes focuses on teaching prosocial skills and behaviors and positive social problem solving. We know that antisocial behavior problems are more resistant to change [13] and that positive impact on antisocial behavior problems may take longer than the 4-week intervention period of this study. It may be that declines in these antisocial behavior problems would emerge with greater exposure toHall of Heroes as well as greater real-world experience using prosocial skills that are antithetical to antisocial behaviors.With regard to thebehavioral dysfunctionof the Y-OQ, this subscale focuses on ADHD-related symptoms, such as disorganization, poor concentration, and trouble handling frustration, which are not directly addressed by theHall of Heroes intervention. Thus, it is not too surprising that there was no significant group difference for this particular subscale. However, whileHall of Heroes is not designed as a treatment tool for any specific disorder (e.g., Attention Deficit Hyperactivity Disorder, Conduct Disorder), it is possible that improvements in these types of behavioral problem areas may arise over time as youth practice new social emotional skills in the real-world. Future longitudinal research would be needed to test whether group differences in antisocial and ADHD-related behavior problems emerge over time for youth.


## 3.3. Hall of Heroes Evaluation

BecauseHall of Heroes was designed to be used independently by youth and because greater engagement in intervention is associated with greater skill gains (Clark et al., 2016) [47], it was important to examine how youth evaluated the game. In general, as can be seen in Table 4, both youth and parents ratedHall of Heroes favorably across all areas. Parents supportedHall of Heroes as engaging, and youth reported they found the game fun and enjoyed both the characters and story. Also of note were youths’ high marks for interest in playing more games likeHall of Heroes and parents’ belief thatHall of Heroes is an effective tool for teaching social skills with youth.Table 4
Means (SD) of youth and parent evaluations ofHall of Heroes.
Youth Ratings Mean (SD) Overall, I likedHall of Heroes. 4.23 (.83) I would like to play more games likeHall of Heroes. 4.38 (.65) I thoughtHall of Heroes was fun to play. 4.31 (.85) I liked the characters inHall of Heroes. 4.23 (.92) I liked the pictures/graphics inHall of Heroes. 3.92 (1.38) I liked the story inHall of Heroes. 4.15 (1.14)  Parent Ratings Mean (SD)  Hall of Heroes is of high overall quality. 4.00 (.65) Hall of Heroes was fun and engaging for my child. 4.27 (.80) Hall of Heroes was easy for my child to understand. 4.00 (.85) My child’s overall experience withHall of Heroes was positive. 4.13 (.83) Hall of Heroes is an effective tool for teaching students social skills. 4.20 (.68) Hall of Heroes is an effective tool for preparing students for middle school. 4.27 (.59) I would recommendHall of Heroes to other parents of 5th or 6th grade students. 4.33 (.82) Note. All ratings were made on a 5-point scale from 1 (strongly disagree) to 5 (strongly agree).Follow-up requests for open-ended feedback helped to put these ratings into context. When the youth were asked what they learned inHall of Heroes, nearly all noted at least one of the program goals, including “cooperation with others,” “respect for others,” “how to deal with feelings,” and “communication and working together,” to name a few. Representative comments from parents were like “my child learned about working together for a common goal,” “my child had fun enough to keep her wanting to play, yet helped her with some new ideas about how to interact with others,” and “there were more than few scenarios that my child encountered that forced him to reevaluate and change directions,” among other comments. Further, parents reported they were likely to recommendHall of Heroes to other parents of adolescents.


## 4. Conclusion

The results of this study provide support for the usability and acceptability ofHall of Heroes as well as initial evidence of its potential as a game-based social skills training (SST) intervention for adolescents. High ratings from both youth and parents, and the positive open-ended feedback, support the idea that the digital gameHall of Heroes is an acceptable and engaging way to implement SST. Also encouraging was the fact that we found all participants were able to complete the intervention in the allotted time, and there were no reports of technical or user errors with the software. These findings indicate youth are able to completeHall of Heroes independently and are sufficiently engaged in the program to maintain interest and motivation throughout SST.Findings also provided initial support for the efficacy of participation inHall of Heroes for increased interpersonal and emotional strength and decreased psychosocial distress, key ingredients to preparing youth for the transition to middle school. Despite the small sample size and brief intervention period, youth who completed the game-based SST showed improvements in multiple areas. In fact, those areas found to change significantly as a function of intervention (e.g., lower intrapersonal distress, greater interpersonal and affective strengths) showed moderate to high effect sizes, a treatment benefit commiserate with more traditional in-person SST programs (see [13]).This work supports the growing body of evidence showing that digital games are potentially effective for delivering a wide array of psychotherapeutic interventions, including SST. As evidence mounts confirming the importance of social skills as a protective force in early adolescence, particularly in helping youth cope with the stressful transition to middle school, the need for effective intervention methods that can overcome the barriers to traditional SST programs becomes increasingly urgent.Hall of Heroes and programs like it which deliver dynamic, personalized, and engaging social skills instruction offer a low-cost, accessible, and easy to scale alternative to reach youth with effective SST on a broader scale than ever before.


### 4.1. Limitations and Future Directions

This initial study of theHall of Heroes game-based SST intervention adds to the growing literature indicating digital games can effectively address a wide array of social, emotional, and behavioral concerns (see review in [21]). This pilot test represents the first step in evaluating the efficacy ofHall of Heroes specifically and as such a number of limitations should be considered.First, our sample size was quite small which limits statistical power and our ability to investigate potentially informative subgroup differences (e.g., grade or gender differences) and also can potentially limit generalizability of findings. It is important to note that while the sample was recruited from the broader community with no requirement that youth have a preexisting social skills problem, it appears that parents who signed up were at least concerned about their child’s social skills or social functioning. Examination of baseline levels of each outcome measure showed elevated distress in all areas of the Y-OQ and below average strengths on the BERS-2 for both TX and WLC youth (no group differences were present at baseline). Given what appears to be a self-selection bias by parents for participation in this intervention study, results should be considered generalizable to adolescents in need of social skills intervention, rather than youth more generally. Future studies employing a larger sample size are needed to confirm treatment effects and investigate potential differential treatment response by subgroup, including youth with varying degrees of baseline social difficulties.Second, the current study relied on parent report for outcome measures. Given our design, it was not possible to include independent measures where the reporter was not aware of the intervention condition of youth. Because parents were not blind to treatment condition, biases may have influenced their reported behavioral changes. However, it is important to note that the pattern of results was not equivalent across all subscales. If biases were solely responsible for parental ratings, we would expect all subscales to be equally impacted. However, those subscales that were most closely related to the content taught throughHall of Heroes (prosocial skills, positive social problem solving) were those that were found to be significantly improved by participation in the intervention. Outcome areas which are not directly targeted by the intervention (e.g., school-related performance, antisocial behavior problems, and attentional problems) or which may require a longer time period than 4 weeks to see change were not found to be significant in this study. Future investigations with multiple informants and informants who are blind to treatment condition, such as teachers, as well as a longer longitudinal design, would be important for clarifying the observed changes by parents in the current study.Third, the brief intervention period meant data collection and intervention delivery occurred during the spring semester for all participants. Further research is needed to examine whether time of year impacts treatment effects. Studies which compare timing relative to the transition to middle school (e.g., late 5th grade, early/late 6th grade) and include measurement beyond immediate postintervention would be useful for determining the optimal conditions for delivering the intervention.


## 4.1. Limitations and Future Directions

This initial study of theHall of Heroes game-based SST intervention adds to the growing literature indicating digital games can effectively address a wide array of social, emotional, and behavioral concerns (see review in [21]). This pilot test represents the first step in evaluating the efficacy ofHall of Heroes specifically and as such a number of limitations should be considered.First, our sample size was quite small which limits statistical power and our ability to investigate potentially informative subgroup differences (e.g., grade or gender differences) and also can potentially limit generalizability of findings. It is important to note that while the sample was recruited from the broader community with no requirement that youth have a preexisting social skills problem, it appears that parents who signed up were at least concerned about their child’s social skills or social functioning. Examination of baseline levels of each outcome measure showed elevated distress in all areas of the Y-OQ and below average strengths on the BERS-2 for both TX and WLC youth (no group differences were present at baseline). Given what appears to be a self-selection bias by parents for participation in this intervention study, results should be considered generalizable to adolescents in need of social skills intervention, rather than youth more generally. Future studies employing a larger sample size are needed to confirm treatment effects and investigate potential differential treatment response by subgroup, including youth with varying degrees of baseline social difficulties.Second, the current study relied on parent report for outcome measures. Given our design, it was not possible to include independent measures where the reporter was not aware of the intervention condition of youth. Because parents were not blind to treatment condition, biases may have influenced their reported behavioral changes. However, it is important to note that the pattern of results was not equivalent across all subscales. If biases were solely responsible for parental ratings, we would expect all subscales to be equally impacted. However, those subscales that were most closely related to the content taught throughHall of Heroes (prosocial skills, positive social problem solving) were those that were found to be significantly improved by participation in the intervention. Outcome areas which are not directly targeted by the intervention (e.g., school-related performance, antisocial behavior problems, and attentional problems) or which may require a longer time period than 4 weeks to see change were not found to be significant in this study. Future investigations with multiple informants and informants who are blind to treatment condition, such as teachers, as well as a longer longitudinal design, would be important for clarifying the observed changes by parents in the current study.Third, the brief intervention period meant data collection and intervention delivery occurred during the spring semester for all participants. Further research is needed to examine whether time of year impacts treatment effects. Studies which compare timing relative to the transition to middle school (e.g., late 5th grade, early/late 6th grade) and include measurement beyond immediate postintervention would be useful for determining the optimal conditions for delivering the intervention.



---

*Source: 6981698-2019-01-01.xml*