# A Revisit of the Measurements on Engagement in Videogames: A New Scale Development

Amir Zaib Abbasi, Ding Hooi Ting, Helmut Hlavacs

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Amir Zaib Abbasi, Ding Hooi Ting, Helmut Hlavacs. A Revisit of the Measurements on Engagement in Videogames: A New Scale Development. 15th International Conference on Entertainment Computing (ICEC), Sep 2016, Wien, Austria. pp.247-252, 10.1007/978-3-319-46100-7_25. hal-01640282

HAL Id: hal-01640282

https://inria.hal.science/hal-01640282

Submitted on 20 Nov 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Revisit of the Measurements on Engagement in Videogames: A New Scale Development

Amir Zaib Abbasi¹, Ding Hooi Ting¹, and Helmut Hlavacs²

¹Department of Management and Humanities, Universiti Teknologi PETRONAS, Tronoh, Malaysia

amir_zaib_abbasi@yahoo.co.uk

²Research Group Entertainment Computing, University of Vienna

**Abstract.** This research article attempts to conceptualize and operationalize the concept of engagement in videogame-play as consumer videogame engagement that comprises both psychological and behavioral dimensions. Accordingly, this study has developed a scale for measuring consumer videogame engagement through following the steps of scale development. Next, the study has collected data on two samples. Besides, this study has applied SPSS 22.0 version and SEM-PLS approach to analyze the data on two samples and to validate the construct of consumer videogame engagement. Based on the study results, an instrument has proven to be a valid source for measuring engagement in videogames as well as a reflective-formative and multi-dimensional construct. This study contributes to the videogame literature as it considers consumer videogame engagement as a multi-dimensional construct comprising on cognitive, affective and behavioral engagement. It further validates the scale of consumer videogame engagement as reflective-formative model among videogame players.

**Keywords:** Engagement, videogame engagement, consumer videogame engagement, cognitive engagement, affective engagement, behavioral engagement, reflective-formative model, scale development process.

## 1 Introduction

Literature indicates that several studies have put their efforts to develop an instrument to measure engagement in videogame playing. Among all the studies, a study by [1], was the first research who developed an instrument to measure the subjective experience of videogame-play based on these dimensions "gameflow, cognitive-absorption and presence" and named it as game-experience questionnaire or GEQ. Another subsequent study done by [2] who developed a scale named as game engagement questionnaire or GEQ with combination of four constructs "flow, presence, absorption and immersion" to measure player's engagement in videogame. Again, this study has also used almost similar dimensions as it was used in the following study [1]. In similar vein, a recent study [3] has reviewed the model of game engagement construct given by [2]. In this study, the author has discussed that three of the dimensions of game
engagement questionnaire such as "flow, immersion and presence" [2] are interrelated to the subjective experience of videogame-play but absorption is similar to flow construct so, it is better not to consider in the measurement of game-engagement questionnaire. According to [3], a new revised construct of game-engagement comprises the following dimensions "flow, presence, involvement and immersion" that can be applied to examine the subjective experience of videogame play.

Doing a critical review of prior researches that have developed and used engagement scale in their studies. The current study has concluded with the following limitations; first, this study finds that mostly studies have only considered psychological constructs (flow, presence and absorption) and have ignored the importance of using behavioral dimensions to examine the subjective experience of videogame play as well as measure the player's engagement in video games [1-3]. Secondly, these studies have not properly conceptualized and operationalized the term engagement and immersion in their studies.

Moreover, several studies have stated that engagement in videogame, is a multi-dimensional construct [4, 5]. Among these studies, only a study by [4] who really captured the meaning of engagement on a multi-dimensional level. But, the problem with the following study [4] is, authors have misspecified the construct of game immersion as reflective-reflective construct (A reflective measurement model is; when items share a common theme, items are exchangeable and deleting or adding an item does not change the conceptual meaning of the construct [6]) on first and second order construct. Rather, the construct of game immersion seems to be as reflective-formative construct that means reflective on first order and formative on second order construct (A formative measurement is; when items do not require to have a shared common theme, items are not replaceable and deleting and adding an item can change the conceptual meaning of the construct [6]). The decision behind calling a game immersion construct as reflective-formative was drawn on basis of reviewing the following study [6] that has discussed the important decision rules for specifying the construct as reflective or formative.

Therefore, the current study considers the limitations of past studies and takes an initiative to first conceptualize and operationalize the term engagement in videogames as "consumer video-game engagement" and develops a new scale with combination of psychological as well as behavioral dimensions in order to measure engagement in videogame-playing in more advanced level. Moreover, this study aims to validate a newly developed construct among the videogame consumers. The present study is first in videogame literature which considers both psychological and behavioral dimensions for measuring consumer video-game engagement. Secondly, this study considers the importance of both reflective and formative constructs in explaining and measuring the construct of consumer videogame engagement on higher order construct. Thirdly, this study applies the concept of marketing and consumer research, especially the definition of consumer engagement and its related dimensions (cognitive, affective and behavioral) that have been discussed in the following studies [7, 8] in order to conceptualize and operationalize the construct of consumer videogame engagement.

## 2. Conceptualization of Consumer Videogame Engagement
This study follows the definition of engagement given by [7, 8] as "*Engagement is a multidimensional construct which is subject to a context-specific expression of relevant cognitive, emotional, and behavioral dimensions*". The author of the study [8], has further added that engagement is a process which reveals as a result of two-way communications between the engagement-subject (consumer/customer) and a particular engagement-object such as a product, service or a brand, which leads to generate consumer engagement states (*cognitive, affective and behavioral*). On the basis of above stated definition, this study conceptualizes consumer videogame engagement as "*A psychological state that triggers due to two-way interactions between the consumer and videogame product, which generates different level of consumer engagement states (cognitive, affective and behavioral)*".

### 3. Scale Development and the Validation of Consumer Videogame Engagement

This study adopts the scale development procedure given by [9, 10]. This study first compiles the measurement scales as already reported in the literature which is specifically relevant to the following dimensions: conscious-attention, absorption (cognitive engagement), dedication, enthusiasm (affective engagement), social interaction and interaction (behavioral engagement). The items on the following dimensions such as conscious attention, six items (Items 1 to 6) from [11], item7 from [12] and item8 from [13] are adapted in this study. Absorption scale adapted from [14]. Whereas, dedication items as item1 from [15] and item2 to item7 from [16] and the items of enthusiasm are adapted from the study of [11]. However, social connection scale is adapted as item1 to item4 from [11] and remaining items 5,6 and 7 from [17]. Finally, interaction scale is adapted from [14]. Overall, the study has generated 39 items, which were further evaluated for the content validity by 4 PhD students and two experts to assess the content of consumer videogame engagement. Based on their comments, this study has revised certain items.

For first-time data collection, 200 questionnaires were distributed among university teenage students who play videogames aged 16-19, out of which 160 questionnaires were returned with a response rate of 80%. Out of 160 questionnaires, 134 valid cases were left after treating the missing values and biased responses. The data were further analyzed for exploratory factor analysis (EFA) as well as for internal consistency and the results are shown in the Table1. After finding the satisfactory results, this study continued to second-time data collection, 350 questionnaires were distributed and collected 290 responses with response rate of 83%. After treating missing values and biased responses, we were left with 265 valid cases that were used for further analysis.

SEM-PLS (*Structural equation modeling-partial least squares*) was employed to validate the construct of consumer videogame engagement because, it has both reflective and formative constructs [18]. For reflective construct, this study involves the estimation of internal consistency, convergent validity and discriminant validity. Whereas, the evaluation of a formative construct follows different guidelines like testing of multicollinearity, indicator weights, and redundancy analysis.

Initially, a reflective measurement model was assessed for its convergent validity and achieved confirmatory factor loadings (CFA) greater than 0.70, composite reliability ranged from 0.89 to 0.92, while AVE were in between 0.61 to 0.75. All
criteria of convergent validity were met. Discriminant validity was evaluated through newly introduced method called Heterotrait-Monotrait ratio (HTMT) of correlations [19] and found that all values have passed the critical value of HTMT .85 as in Table 2.

This study has assessed the three criteria for the evaluation of formative measurement model as suggested by [20]. The first criteria is to assess the convergent validity also termed as redundancy analysis. The results indicate that the path coefficient between consumer video-game engagement and global measure is 0.826, which is higher than the threshold value of 0.80 as recommended by [20] and thus, depicting that the formative constructs have achieved the convergent validity. Second, the study has checked the collinearity issues and found that there is no multicollinearity issue because, all values shown in the Table 3, are below the threshold value of 5 as suggested by [20]. Third, this study has assessed the significance of indicator weights as shown in the Table 3 and found that formative constructs are highly significant at 0.00 level.

## 4. Conclusion

This study first conceptualized operationalized the construct of consumer videogame engagement then developed a scale for measuring consumer video-game engagement. The scale was first validated through EFA and reliability test. Second, the data were recollected on more respondents for measuring CFA and reflective measurement model on first order. All results met the criteria for the assessment of reflective measurement model then second order were created on a formative level. Formative measurement model was assessed on basis of three criteria given by [20] and results met the criteria for the evaluation of a formative model.

<table><caption>Table: 1 Shows Results of Both First and Second Time Data Collection</caption><thead><tr><th rowspan="2">Construct</th><th rowspan="2">Items</th><th colspan="2">Study one: N=134</th><th colspan="3">Study two (N=265): Convergent Validity</th></tr><tr><th>EFA Loadings</th><th>C.Alpha</th><th>CFA Loadings</th><th>C.Alpha</th><th>CR</th><th>AVE</th></tr></thead><tbody><tr><td rowspan="5">Absorption</td><td>When I am playing this video-game, I forget everything else around me.</td><td>0.82</td><td>0.83</td><td>0.73</td><td>0.84</td><td>0.89</td><td>0.61</td></tr><tr><td>Time flies when I am playing this video-game.</td><td>0.63</td><td></td><td>0.73</td><td></td><td></td><td></td></tr><tr><td>When I am playing this video-game, I get carried away.</td><td>0.93</td><td></td><td>0.83</td><td></td><td></td><td></td></tr><tr><td>It is difficult to detach myself from playing this video-game.</td><td>0.82</td><td></td><td>Deleted</td><td></td><td></td><td></td></tr><tr><td>When I am playing this video-game, I feel immersed.</td><td>0.48</td><td></td><td>0.80</td><td></td><td></td><td></td></tr><tr><td></td><td>I feel happy, when I am playing this video-game intensely.</td><td>Deleted</td><td></td><td>0.81</td><td></td><td></td><td></td></tr><tr><td rowspan="5">Conscious Attention</td><td>I like to know more about this video-game.</td><td>0.87</td><td>0.89</td><td>Deleted</td><td></td><td></td><td></td></tr><tr><td>I like to learn more about this video-game.</td><td>0.92</td><td></td><td>0.82</td><td>0.88</td><td>0.92</td><td>0.74</td></tr><tr><td>I notice information related to this video-game.</td><td>0.83</td><td></td><td>0.85</td><td></td><td></td><td></td></tr><tr><td>I pay a lot of attention to anything about this video-game.</td><td>0.74</td><td></td><td>0.88</td><td></td><td></td><td></td></tr><tr><td>I keep up with things related to this video-game.</td><td>0.65</td><td></td><td>0.88</td><td></td><td></td><td></td></tr><tr><td rowspan="5">Dedication</td><td>This video-game inspires me.</td><td>0.71</td><td>0.84</td><td>0.84</td><td>0.86</td><td>0.91</td><td>0.71</td></tr><tr><td>I am enthusiastic about playing this video-game.</td><td>0.85</td><td></td><td>0.85</td><td></td><td></td><td></td></tr><tr><td>I am proud of playing this video-game.</td><td>0.65</td><td></td><td>0.84</td><td></td><td></td><td></td></tr><tr><td>I find this video-game full of meaning and purpose.</td><td>0.84</td><td></td><td>0.84</td><td></td><td></td><td></td></tr><tr><td>I am excited when playing this video-game.</td><td>0.82</td><td></td><td>0.82</td><td></td><td></td><td></td></tr></tbody></table>
<table><caption>Table 2: Discriminant Validity (HTMT Ratio)</caption><tbody><tr><td rowspan="5">Enthusiasm</td><td>I spend a lot of my discretionary time playing this video-game.</td><td>0.8</td><td>0.85</td><td>0.75</td><td>0.86</td><td>0.90</td><td>0.65</td></tr><tr><td>I am heavily into playing this video-game.</td><td>0.91</td><td></td><td>0.85</td><td></td><td></td><td></td></tr><tr><td>I am passionate about playing this video-game.</td><td>0.67</td><td></td><td>0.84</td><td></td><td></td><td></td></tr><tr><td>I enjoy spending time playing this video-game.</td><td>0.52</td><td></td><td>0.81</td><td></td><td></td><td></td></tr><tr><td>I try to fit playing this video-game into my schedule.</td><td>0.85</td><td></td><td>0.77</td><td></td><td></td><td></td></tr><tr><td rowspan="5">Interaction</td><td>In general, I like to get involved in the discussions about this video-game playing.</td><td>0.75</td><td>0.88</td><td>0.83</td><td>0.88</td><td>0.91</td><td>0.68</td></tr><tr><td>I am someone who enjoys playing this video-game with others like-minded video-game players.</td><td>0.73</td><td></td><td>0.81</td><td></td><td></td><td></td></tr><tr><td>I am someone who likes actively participating in the discussions about this video-game playing.</td><td>0.95</td><td></td><td>0.87</td><td></td><td></td><td></td></tr><tr><td>In general, I thoroughly enjoy exchanging ideas on this video-game with other video-game players.</td><td>0.84</td><td></td><td>0.82</td><td></td><td></td><td></td></tr><tr><td>I often participate in activities relating to this video-game.</td><td>0.77</td><td></td><td>0.79</td><td></td><td></td><td></td></tr><tr><td rowspan="3">Social Connection</td><td>I love playing this video-game with my friends.</td><td>0.91</td><td>0.86</td><td>0.89</td><td>0.83</td><td>0.90</td><td>0.75</td></tr><tr><td>I enjoy playing this video-game more when I am with others.</td><td>0.91</td><td></td><td>0.85</td><td></td><td></td><td></td></tr><tr><td>Playing this video-game is more fun when other people around me play it too.</td><td>0.88</td><td></td><td>0.86</td><td></td><td></td><td></td></tr><tr><td></td><td>I receive social fulfillment from playing this video-game.</td><td>0.59</td><td></td><td>Deleted</td><td></td><td></td><td></td></tr></tbody></table>

<table><caption>Table 3: Evaluation of Formative Constructs</caption><thead><tr><th></th><th>Absorp</th><th>ConsAtten</th><th>Dedicate</th><th>Enthusi</th><th>Interact</th><th>SocialConn</th></tr></thead><tbody><tr><td>Absorp</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>ConsAtten</td><td>0.683</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Dedicate</td><td>0.656</td><td>0.836</td><td></td><td></td><td></td><td></td></tr><tr><td>Enthusi</td><td>0.806</td><td>0.73</td><td>0.697</td><td></td><td></td><td></td></tr><tr><td>Interact</td><td>0.582</td><td>0.618</td><td>0.654</td><td>0.661</td><td></td><td></td></tr><tr><td>SocialConn</td><td>0.414</td><td>0.576</td><td>0.46</td><td>0.561</td><td>0.621</td><td></td></tr></tbody></table>

<table><thead><tr><th>Construct Scale</th><th>Measurement model</th><th>Items</th><th>Weights</th><th>Full Collinearity (VIFs)</th><th>T-Values</th><th>P-Values</th></tr></thead><tbody><tr><td rowspan="2">Cognitive Engagement</td><td rowspan="2">Formative</td><td>Absorption</td><td>0.40</td><td>3.1</td><td>3.16</td><td>0.00</td></tr><tr><td>ConsAtten</td><td>0.71</td><td></td><td>6.91</td><td>0.00</td></tr><tr><td rowspan="2">Affective Engagement</td><td rowspan="2">Formative</td><td>Dedication</td><td>0.50</td><td>3.3</td><td>4.90</td><td>0.00</td></tr><tr><td>Enthusiasm</td><td>0.62</td><td></td><td>6.12</td><td>0.00</td></tr><tr><td rowspan="2">Behavioral Engagement</td><td rowspan="2">Formative</td><td>Interaction</td><td>0.78</td><td>1.7</td><td>10.15</td><td>0.00</td></tr><tr><td>Social connection</td><td>0.34</td><td></td><td>3.51</td><td>0.00</td></tr></tbody></table>
5. References

1. Jennett, C., et al., *Measuring and defining the experience of immersion in games*. International journal of human-computer studies, 2008. 66(9): p. 641-661.

2. Brockmyer, J.H., et al., *The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing*. Journal of Experimental Social Psychology, 2009. 45(4): p. 624-634.

3. Procci, K.C., *The Subjective Gameplay Experience: An Examination of the Revised Game Engagement Model (PhD Diss)*. 2015, University of Central Florida Orlando, Florida.

4. Cheng, M.T., H.C. She, and L.A. Annetta, *Game immersion experience: its hierarchical structure and impact on game-based science learning*. Journal of Computer Assisted Learning, 2015. 31(3): p. 232-253.

5. Silpasuwanchai, C., et al. *Developing a Comprehensive Engagement Framework of Gamification for Reflective Learning*. in *Proceedings of the 2016 ACM Conference on Designing Interactive Systems*. 2016: p. 459-472.

6. Jarvis, C.B., S.B. MacKenzie, and P.M. Podsakoff, *A critical review of construct indicators and measurement model misspecification in marketing and consumer research*. Journal of consumer research, 2003. 30(2): p. 199-218.

7. Brodie, R.J., et al., *Customer engagement: conceptual domain, fundamental propositions, and implications for research*. Journal of Service Research, 2011. 14(3): p. 252-271.

8. Hollebeek, L.D., *Demystifying customer brand engagement: Exploring the loyalty nexus*. Journal of Marketing Management, 2011. 27(7-8): p. 785-807.

9. Churchill Jr, G.A., *A paradigm for developing better measures of marketing constructs*. Journal of marketing research, 1979: p. 64-73.

10. Tsaur, S.-H., C.-H. Yen, and Y.-T. Yan, *Destination brand identity: scale development and validation*. Asia Pacific Journal of Tourism Research, 2016: p. 1-14.

11. Vivek, S.D., et al., *A generalized multidimensional scale for measuring customer engagement*. Journal of Marketing Theory and Practice, 2014. 22(4): p. 401-420.

12. Qin, H., P.-L. Patrick Rau, and G. Salvendy, *Measuring player immersion in the computer game narrative*. Intl. Journal of Human-Computer Interaction, 2009. 25(2): p. 107-133.

13. Wiebe, E.N., et al., *Measuring engagement in video game-based environments: Investigation of the User Engagement Scale*. Computers in Human Behavior, 2014. 32: p. 123-132.

14. So, K.K.F., C. King, and B. Sparks, *Customer engagement with tourism brands scale development and validation*. Journal of Hospitality & Tourism Research, 2014. 38(3): p. 304-329.

15. Schaufeli, W.B., et al., *The measurement of engagement and burnout: A two sample confirmatory factor analytic approach*. Journal of Happiness studies, 2002. 3(1): p. 71-92.

16. Cheung, C.M., et al., *Promoting sales of online games through customer engagement*. Electronic Commerce Research and Applications, 2015. 14(4): p. 241-250.

17. Kemp, E., *Engaging consumers in esthetic offerings: conceptualizing and developing a measure for arts engagement*. International Journal of Nonprofit and Voluntary Sector Marketing, 2015. 20(2): p. 137-148.

18. Hair, J.F., C.M. Ringle, and M. Sarstedt, *PLS-SEM: Indeed a silver bullet*. Journal of Marketing theory and Practice, 2011. 19(2): p. 139-152.

19. Henseler, J., C.M. Ringle, and M. Sarstedt, *A new criterion for assessing discriminant validity in variance-based structural equation modeling*. Journal of the Academy of Marketing Science, 2015. 43(1): p. 115-135.

20. Hair Jr, J.F., et al., *A primer on partial least squares structural equation modeling (PLS-SEM)*. 2013: Sage Publications.