Assessing the Necessary Skill Profiles for Playing Video Games

It seems clear that different video games require different skills. However, there has been no systematic way of assessing what these skills are or for assessing the extent to which particular skills are required by a particular game. This study used a psychometric approach to help identify these skills and profile particular games and genres of video games. Experienced gamers generated a list of 32 skills and then a diverse sample of participants rated a number of games on the extent to which they required the skills. Factor analysis revealed seven general components: perceptual-motor, role-playing, numerical reasoning, problem solving, focuspersistence, acceptance of uncertainty, and player interaction. Different genres of games differed significantly on a number of these components. The resulting instrument can be used by the game industry to profile games for review and evaluation. INTRODUCTION It has been known since the inception of video games that each requires different skills to play the game. Often these skills involve perceptual speed, targeting and reaction time, but other games require strategic thinking and cognitive processing of information. While games require different skills, there is yet no definitive list of these skills or method to evaluate the skills required by a particular game. This study was conducted to develop an instrument that could be used by individuals familiar with a particular video game to assess the various skills required to play the game. Review of the Literature Previous research has identified some of the skills involved in playing different video games. For example, Dorval and Pepin (1986) found that subjects who played 8 sessions of Zaxxon, in which the player controls a spaceship in a threedimensional space and attempts to shoot enemies and avoid obstacles and being hit, showed significantly higher spatial skill scores than the control group. A number of studies have been primarily interested in using video games to develop skills that would transfer to other contexts. For example, Mulligan, Dobson, and McCracken (2005) looked at visual processing skills developed by video game players and hockey players. The video game ThinkHockey was used to teach players the strategic and Human-Computer Interaction Technical Report 12-31-16 2 tactical aspects of the game. The skills would then transfer over to real-life hockey. Other examples of video games that develop skills to transfer over would be the use of Madden video games by NFL athletes, as well as targeting and shooting games by the military. These studies identify a skill of interest and then attempt to find a video game that could be used to develop this skill. Other studies have looked at the effect of videogame playing in general on the development of particular skills. Green and Bavelier (2004) summarize research on the effects of playing video games on reaction time and perceptual-motor coordination, spatial skills, and visual attention. Griffith et al. (1983) found that video game players far outperformed nonplayers on a rotary pursuit task, especially at high speeds. In a controlled study, Orosy-Fildes and Allan (1989) found that when half of the subjects underwent a 15-minute practice on an Atari 2600 video game system, they displayed a faster reaction time than the control group. Recent research provides evidence for improved skills with more modern gaming systems as well. Strobach, Frensch, and Schubert (2012) found that video game play improved dual task and task switching skills in both regular gamers and nongamers. Furthermore, Feng, Spence, and Pratt (2007) determined that spatial attention and mental rotation improved after playing 10 hours of an action game. When examining general attention skills, Dye, Green, and Bavelier (2009) found that action video gamers performed significantly better on the Attentional Network Test (ANT) than non-action video gamers. This reveals that action video game players have enhanced attention skills compared to their nonaction video game player counterparts. We see that these effects extend even to younger children. In a study done by Masfety et al. (2016), high video game usage of more than five hours a week was significantly associated with higher intellectual functioning and increased academic achievement. However, these studies have generally identified changes in skills as a function of either playing video games in general or specific games without first identifying the skills required. To identify all of the skills needed to play a particular video game, a different approach is required. One method is to perform a task analysis of a game and then relate the tasks to the skills. Human factors psychology has developed procedures for task analyses (e.g., Militello, Hutton, Pliske, Knight, & Klein; 1997). While this could be done for video games, it would be extremely laborious. Moreover, video games today can be extremely complex, making an exhaustive task analysis nearly impossible. Moreover, players play games in different ways using different skill sets, so the skills identified in the task analysis may not necessarily relate to the skill sets or abilities of all players. Similar to task analyses, laboratory methods measuring specific skills developed following game play or correlated with game performance are also too time consuming and expensive to be of use considering the thousands of games that one would like to assess. Another method is to conduct a conceptual analysis of skills learned. Smith (2008) proposes a method for identifying the skills developed in video games in a manner similar to identifying the skills involved in college courses, such as Human-Computer Interaction Technical Report 12-31-16 3 rhetorical and compositional skills. She bases this on Johnson’s (2005) discussion in Everything Bad is Good for You on how students think and write critically about their experiences playing video games. This conceptual approach of using player assessments from their experiences playing games seems more viable than either task analyses or performance assessments. Consequently, the approach taken by Norman (2010) was to use psychometric methods to develop a set of scales that could be used to evaluate games. To develop the scale, experienced gamers created a pool of questions. These and other gamers served as coders to rate games that they were familiar with on the extent to which the game required each of these skills or abilities. The current study expands upon this work by including a larger pool of participants and video games. The previous scale was modified to include two more items to better capture the diverse skills required in games. METHOD Participants Four different sets of coders participated in this study. The first group consisted of upper level undergraduate students in a course on the “Psychology of Video Games and Entertainment” from 2011-2015 at the University of Maryland. Each student rated 5 games that they were familiar resulting in n = 929. Only a portion of the coders provided demographics information. About 44% of these identified as female. The second group (n = 236) consisted of undergraduate student participants recruited through the psychology department SONA system. The participants’ racial selfidentifications are as follows: 55% Caucasian, 21% Asian, 14% African, 9% Hispanic, 5% Other or Multiracial. Approximately 56% of participants self-identified as male and 44% selfidentified as female. Approximately 39% of participants identified as “casual” gamers, 20% as moderate plus, 19% as moderate, 8% as nongamers, 7.5% as heavy gamers, and 1% as hardcore gamers. The final group (n = 195) consisted of a “convenience sample” who found and took the skills questionnaire through links to the laboratory website. Demographics were not available for this group. Rating Scale The items used for this questionnaire were originally discussed and generated by students participating in a one day a week Summer Video Game Internship on the psychology of video games, held from June 2 to July 28, 2010. The rating form consisted of 24 items (Norman, 2010). After the scale was piloted, 8 additional items were added. The current set is shown in Appendix A. The items consisted of a stem listing the skill or ability required of player for the game and a 9-point scale with endpoints “Not necessary” and “Very necessary.” The name of the game being rated, and any additional comments that they had about the game were also recorded. The ratings were submitted on the Web and stored in a FileMaker Pro database. Human-Computer Interaction Technical Report 12-31-16 4 RESULTS A total of 1360 ratings of various games were generated by the participants. Of these, 1320 where complete enough to be used in the data analysis. In all, 293 series of games were rated. Factor Analysis An exploratory factor analysis (principal components with a Varimax orthogonal rotation) was conducted on the data. Seven factors were identified. Appendix B shows the table of factor loadings for the 32 items. The seven factors were be labeled as follows: Factor 1: PerceptualMotor, Factor 2: Role-Playing, Factor 3: Numerical Reasoning, Factor 4: Problem Solving, Factor 5: Focus-Persistence, Factor 6: Acceptance of Uncertainty, and Factor 7: Player Interaction. Reliability Cronbach’s Alpha was .895 for all 32 items collectively. The Cronbach’s Alpha for each subscale were: Perceptual-Motor, .826; RolePlaying, .844; Numerical Reasoning, .858; Problem-Solving, .610; Focus-Persistence, .739; Acceptance of Uncertainty, .411; and Player Interaction .490. Cumulatively, the scale is in the respectable range. Compared to Alpha = .7, the subscales of Perceptual-Motor, Role-Playing, Numerical Reasoning, and Focus-Persistence are deemed to be acceptable. Problem Solving, Acceptance of Uncertainty, and Player Interaction may be less than acceptable. Factor Variates as a Function of Game The following 12