# Raising Students' Cognitive Engagement Intention in a Preliminary IS Course Using Gamification

Mathupayas Thongmak

► To cite this version:

Mathupayas Thongmak. Raising Students' Cognitive Engagement Intention in a Preliminary IS Course Using Gamification. 12th International Conference on Research and Practical Issues of Enterprise Information Systems (CONFENIS), Sep 2018, Poznan, Poland. pp.81-95, 10.1007/978-3-319-99040-8_7. hal-01963061

HAL Id: hal-01963061

https://inria.hal.science/hal-01963061

Submitted on 21 Dec 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Raising Students' Cognitive Engagement Intention in a Preliminary IS Course Using Gamification

Mathupayas Thongmak [0000-0001-9064-2071]

Thammasat Business School, Thammasat University, Bangkok 10200, TH
mathupayas@tbs.tu.ac.th

**Abstract.** This paper guides factors influencing undergraduate students' cognitive engagement intention, using activities in the preliminary Information Systems (IS) course as the gamified settings. A simple process, using student-led activities, to implement gamification in educational environments, is described. A research model, consisting of game elements, perceived game usefulness, attitude towards the course, and cognitive engagement intention, is proposed. The main objectives are to examine the impact of these factors on students' cognitive engagement intention. Data are collected using questionnaires. The model is statistically tested using structure equation modeling. The results show that game elements directly and indirectly affect cognitive engagement intention. Student's attitude towards the course, which is strongly influenced by perceived game usefulness, is also important to raise their cognitive engagement.

**Keywords:** Gamification, perceived usefulness, attitudes, cognitive engagement, IS course

## 1 Introduction

Engagement in the course is important for developing students' full learning potential. However, there is still difficulty in keeping students engaged in in-class activities. In addition, only small groups of students engage themselves in learning activities [1]. In the educational context, games are considered as an influential factor for developing knowledge and engaging students [1]. Adding game elements into those activities possibly make the activities more appealing as well as raising students' level of engagement. These are called gamification [2]. Gamification applications are shown to be effective drivers for participants in various domains such as finance and education [3]. Objectives of gamification in education are to stimulate students' motivation and engagement towards games, to improve participation among learners, and to persuade the desired behaviors [4-8]. Fredricks, Blumenfeld and Paris defined three types of engagement comprising of behavioral engagement, emotional engagement, and cognitive engagement [9, 10]. In terms of gamification approaches, they are classified as gamifying learning activity, gamifying social activity, and gamifying assessment activity [5].

Although gamification in education has been explored as described in the Section 2, it as an academic topic that is relatively young [11]. In some countries such as China,
the educational gamification is also in the exploring stage [12]. The effectiveness of gamification for learning is still controversial [13]. More research is needed to determine the long term benefits together with the short term benefits of gamification [2, 8]. Gamified elements are needed to be studied [14]. Motivational drivers influencing students' attitude towards the educational methodology should be identified [15]. Ongoing challenges for designers and practitioners to engage users are the identification and prioritization of engagement factors [16]. Only few well-established theoretical frameworks are presented [11]. Cognitive engagement is motivated behavior [17], which is measured by attention and reflection [13]. It remains under-investigated and inconclusive whether gamification can enable students to think deeply [13]. In addition, gamification is quite complicated to implement [4]. Therefore, the purposes of this study are: 1) to study factors directly and indirectly affecting students' attitudes toward the course, including game elements and perceived usefulness of the games 2) to examine and to prioritize the factors affecting students' cognitive engagement intention, and 3) to guide the simple process of implementing gamification to the course.

## 2 Related Research

The work of Mitchell, Danino and May applied a gamification approach such as challenges, points, and leaderboard to an introductory undergraduate module in Computing. These competitive elements could encourage students to share knowledge and techniques to support less-experienced team members [18]. Cheong, Filippou and Cheong explore undergraduate IT students' perceptions of game elements. Results indicated that students' had positive perceptions of gamified systems and were interested in the use of gamification in learning [4]. Silpasuwanchai, Ma, Shigemasu and Ren proposed a framework of engagement in gamification for learners. The framework showed the connections between gamification strategies, engagement dimensions, and final learning outcomes [13]. Iten and Petko examined the relationship between anticipated enjoyment, willingness to play, game enjoyment, self-reported cognitiveness, and motivational learning gains. Results showed that anticipated enjoyment and actual game enjoyment played a minor role in the students' willingness to learn with serious games [19]. Galbis Córdova, Martí Parreño and Currás Pérez investigated key motivators of undergraduate students' attitude towards the use of gamification as an educational innovation to improve their competencies. Findings pointed that perceived attention, perceived relevance, and perceived confidence positively affected students' attitude towards using online educational video games. Perceived attention and perceived confidence also influenced perceived relevance [15]. Ortiz-Rojas, Chiluiza and Valcke analyzed the effects of gamification (badges) on learning performance, intrinsic motivation, self-efficacy, and engagement of engineering students in a programming course, using a quasi-experiment. Results showed a statistically significant enhancement in students' engagement, compared to the control group, but no significant improvement in learning performance, intrinsic motivation, and self-efficacy [14]. Mahmud, Weber and Moening tried to investigate students' perceptions about badges, experience points tracking with levels, leaderboards, and quizzes with automated feedback implemented
in Moodle and understand how students' learning of the course materials increase through gamification. Results from an online survey showed that game-like elements were slightly helpful towards students' motivation. The combination of experience points with levels was not a strong driver [2]. Putz and Treiblmaier empirically tested the effectiveness of gamification in business context, using gamified workshops. It was found that enjoyment and curiosity were fostered by gamification elements and later influenced individuals' attitudes and behavioral intention to apply sustainable business practices [20].

## 3 Theoretical Background and Hypotheses

<figure><img src="image_2.png" /><figcaption>Fig. 1. Conceptual Framework</figcaption></figure>

Gamified applications are systems use game-designed elements to achieve educational goals by letting students learn by playing a specific game [21]. Perceived usefulness is defined as the perceived increase of understanding, problem solving skills, creativity, and interests in the course topics after conducting games in the class. These attributes such as challenge/problem solving, creativity, social interaction, helping others, importance/utility in business, image/reputation, and change are sub-constructs of the practical application of Management Information Systems (MIS) coursework influencing students when they select their majors [22]. Past studies show the evidence that game-based learning enhances competencies such as critical thinking/decision-making, problem-solving, conflict-resolution, and communication skills [15]. Games enabling children to confront with problems, learn from their mistakes, and put efforts to find a solution, developing problem solving skills and thinking skills for children [23]. One of the anticipated benefits of gamification in the views of students is the improvement of their understanding of the course content [4]. Games provide experiences across var-
engagement framework of gamification for learning. Gamification strategies consisting badges, points, leaderboard, etc., affect engagement (behavioral engagement, emotional engagement, and cognitive engagement), having internal states, tasks, and user characteristics as mediators. The findings show the significant increase of cognitive engagement (attention and reflection) in the gamified group over the non-gamified group [13]. Students' perceived engagement and motivation are increased after the application of gamification [8]. Gamified approach seems to have a positive effect on engagement, for example, the number of downloads, posts, and lecture attendance [30]. Many learners agree that the gamification presence can motivate them to work harder [18]. After completing the gamified modules, the cognitive competence is increased, showing self-efficacy advantages from gamification [31].

Therefore, the following hypothesis is proposed:

*H3: Game elements positively affect students' cognitive engagement intention.*

The perceived usefulness is based on the expectancy theory, involving individuals' believes in their decision making processes [32]. Individuals are driven to engage in an activity if there is a positive expectancy for success [15]. Perceived usefulness significantly create a positive impact on learner's attitude [33]. Gardner and Amoroso [[32] propose that perceived usefulness of the Internet is positively related to attitude towards using the Internet. Higher perceived usefulness leads to the more satisfaction of learners in an e-learning system [34]. Students express their positive attitude towards in-class gamification due to the fun of games [35]. Educational usefulness of gamification has been supported by research of authors who incorporate game mechanisms in their teaching processes [36]. Students play games because of playing with others, the mental challenge, the physical challenge, relieving boredom, and social reasons [4]. Students perceiving online educational video games relevant and believing in the use of online educational video games to develop their competencies significantly express positive attitudes toward the games [15]. Games such as serious games are the active, problem-solving, situated, and social form of learning, which promotes students' enjoyment of learning [19]. Students show their positive attitudes toward gamification because of the game's enjoyment [35]. Enjoyment and curiosity received from gamification significantly influence people's attitudes toward sustainable transport [20].

Therefore, the following hypothesis is proposed:

*H4: Perceived usefulness of gamification positively affects students' attitudes towards the course.*

Perceived usefulness significantly have a positive influence on the intention to engage in gamification of customers [7]. It significantly affects behavioral intention [37] such as in the multimedia ERP training tool [38] and Internet usage [32]. It positively correlates with the continuance intention [39]. Expectations related to usefulness have a significant impact on a child's intention to further work with serious games [19]. Perceived usefulness is used to understand the interaction between ICT learning environments and learners' behavior that are involvement, participation, cognitive engagement, and motivation [33]. Gamification in class provides extra-motivation to students [35]. Improving low student involvement is a potential benefit of gamification in e-
learning [15]. Engagement indicators are autonomy, execution, social, delivery, partic-
ipation, collaboration, cooperation, questioning, organization of the environment and
fun [1]. Cognitive engagement in the academic tasks is the choice that students make
for themselves. It is varied depending on students’ self-defined goals, task purposes,
and their beliefs about the tasks and themselves [17]. Affective and cognitive compo-
nents are the main variables of individual cognitive engagement, which impacts the
information acquisition, information transformation, and knowledge construction [40].
Cognitive engagement in terms of attention significantly correlates with the enjoyment
and excitement aspects of gamification, whereas cognitive engagement in terms of re-
flection significantly correlates with the competence and improvement after the use of
gamification [13].

Therefore, the following hypothesis is proposed:

H5: *Perceived usefulness of gamification positively affects their cognitive engagement intention.*

Engaging students' cognitive reaction to both in-class and outside-class activities is a way to achieve successful learning outcomes [14]. Students believing the interactive and entertaining nature of educational game drawing their attention significantly show their positive attitudes toward the online educational video games [15]. The more positive attitude that a child expresses towards learning games, the more he/she wants to use learning games [19]. Students show the expectation of gamification in learning. They believe that they will in favor of it; they will be excited about it; and it will increase their interest. They also expect the gamification to be better engaging, leading to the improvement of learning [4]. People's attitudes toward sustainable transport using gamification positively affect their intention to use [20]. They are motivated to participate in an activity when it is linked to their satisfaction in the personal value aspect [15].

Therefore, the following hypothesis is proposed:

H6: *Students' attitudes towards the course of gamification positively affect their cognitive engagement intention.*

# 4 Research Methodology

## 4.1 Samples and Setting

The exploratory research was conducted with 209 Thai undergraduate students from five sections, enrolled in the same introductory MIS course in the academic year from 2014 to 2017. The same teacher taught all sections, using the same textbook. The course was the required course (core subjects) for all students who studied in the Thammasat Business School, Thammasat University. They were in the first to the third year of study. Undergraduate students were targeted since they were more likely to play games and had a better understanding of games than postgraduate students [4]. Students were exposed to the experiential activities leading by others and themselves. The experiment
was designed using a simple process to implement to see if the in-class gamified activities would motivate students to cognitively engage in the course. Students were informed about the use of gamification in the classroom, both before and after the gamified activities running. For each section, students were divided into 8 groups. Each group was randomly assigned to conduct one or more games related to one of eight topics. Conducting and playing games as teams also supported students' social beings because students did not like playing alone [4]. After a teacher finished lecturing each course topic, the assigned group ran games in the next period. Proper implementation of gamification should incorporate understanding the players, determining the objectives of the activity, and using appropriate game elements such as achievements, exploration, trial and errors, competition, socialization, and time constraints [1, 2, 4]. Games conducted by students generally included these aspects. When friend groups led the games, students in other groups could choose voluntarily to be game players or game observers. Totally, students participated in 7 games created by peers and 1 game conducted by their group.

## 4.2 Measure, Data Collection, and Data Analysis

The surveys were developed using a 6-point Likert scale to measure game elements, perceived usefulness of games, attitude towards the course, and cognitive engagement intention. Existing scales and definitions of each factor were applied from the literature to develop the questionnaire, as shown in Table 1. The frequency of students' lesson reviews was also collected using a 5-point Likert scale from very rarely to very frequently. Students were surveyed after they conducted one or more games relating to the assigned topic in classes. Of the 210 students from five sections in the consecutive academic year, 209 survey responses were received. The surveys were later analyzed anonymously, using descriptive statistics, confirmatory factor analysis (CFA), and structural equation modeling (SEM). The CFA and SEM are two of the most commonly applied in empirical research and are well recognized in the IS community [20].

<table><caption>**Table 1.** Summary of Constructs and Sources</caption><thead><tr><th></th><th>Example Questions</th><th>Adapted or De-<br>veloped from<br>Sources</th></tr></thead><tbody><tr><td>Game Ele-<br>ments</td><td>Please evaluate the importance of these components when playing games in the classroom:<br>Awards or prizes in games for the winners are creative and varying.<br>Games have a form of communication or interaction between players such as exchange items.<br>(6-point "not at all important – very important)</td><td>[41, 42]</td></tr><tr><td>Perceived Use-<br>fulness of<br>Games</td><td>After conducting the games, what is your level of understanding about this topic?<br>After conducting the games, what is your level of problem solving skills?</td><td>[24, 26, 36,<br>43, 44]</td></tr></tbody></table>
<table><tr><td></td><td>(6-point "lowest – highest)</td><td></td></tr><tr><td>Attitude towards the Course</td><td>I think this course is interesting.<br>I like this course.<br>(6-point "strongly disagree – strongly agree)</td><td>[45]</td></tr><tr><td>Cognitive Engagement Intention</td><td>If games in a classroom composing of various and complete game elements, I will learn more about the lessons if there is in-class game playing.<br>If games in a classroom composing of various and complete game elements, I will study the lessons intensively if there is in-class game playing.<br>(6-point "strongly disagree – strongly agree)</td><td>[46]</td></tr></table>

## 5 Findings

Of 209 students, 123 students were female and 86 students were male. In the first section (1/2014), there were 46 students, who were all freshmen. Of 46 students, 17 students were male and 29 students were female. In the second section (1/2015), there were 28 students. All of them were in the second year of study. There were 13 males and 15 females. In the third section (1/2016/Group1), there were 4 freshmen, 34 sophomores, and 3 junior students. Of 41 students, 20 students were male and 21 students were female. In the fourth section (1/2016/Group2), 43 students consisted of 14 males and 29 females. All of them were in the first year of study. In the last section (1/2017), there were 51 students: 28 freshmen, 21 sophomores, and 2 juniors. Of 51 students, there were 22 males and 29 females. In terms of the frequency of reviewing the lessons (in general), 44 female students (35.8% of total females) reviewed the course content rarely (2-3 times per month). Forty-four females (35.8% of total females) reviewed lessons occasionally (4 times per month). Fifteen female students (12.2% of total females) frequently reviewed the content (2-3 times per week). Eleven females (8.9% of total females) very rarely reviewed the lessons (less than once per month). Nine of them (7.3% of total females) very frequently reviewed the content (more than 4 times per week). Regarding male students, 33 of them (38.4% of total males) rarely reviewed lessons. Twenty of them (23.3% of total males) reviewed the course materials occasionally. Eighteen male students (20.9% of total males) very rarely reviewed the course content. Eleven of them (12.8% of total males) frequently repeated reading course materials. Five male students (4.7% of total males) reviewed lessons very frequently. Table 2 shows the mean and standard deviation of each measurement item.

<table><caption>**Table 2.** Mean and Standard Deviation of Game Elements, Perceived Usefulness, and Attitude towards the Course on Cognitive Engagement Intention</caption><thead><tr><td>Measurement Items</td><td>Mean</td><td>Standard Deviation</td></tr></thead><tbody><tr><td>PU UNDERSTANDING</td><td>5.13</td><td>.685</td></tr><tr><td>PU PROB SOLVE</td><td>4.96</td><td>.774</td></tr></tbody></table>
<table><tr><td>PU_CREATIVITY</td><td>5.02</td><td>.805</td></tr><tr><td>PUaldTINTEREST</td><td>5.01</td><td>.841</td></tr><tr><td>GAME_ELE_VARIETY_REWARD</td><td>4.61</td><td>1.056</td></tr><tr><td>GAME_ELE_AESTHETIC_PROGRESS</td><td>4.57</td><td>.964</td></tr><tr><td>GAME_ELE_PERSONALIZED_AVATAR</td><td>4.09</td><td>1.241</td></tr><tr><td>GAME_ELE_FORMAT_COMMU_EXCHANGE</td><td>4.42</td><td>1.115</td></tr><tr><td>GAME_ELE_SPECIAL_RULE))^</td><td>4.78</td><td>1.023</td></tr><tr><td>ATT_COURSE1</td><td>4.49</td><td>.956</td></tr><tr><td>ATT_COURSE2</td><td>4.29</td><td>.979</td></tr><tr><td>ATT_COURSE3</td><td>4.60</td><td>.956</td></tr><tr><td>COGNITIVE_ENGAGE_INT1</td><td>4.80</td><td>.891</td></tr><tr><td>COGNITIVE_ENGAGE_INT2</td><td>4.70</td><td>.903</td></tr><tr><td>COGNITIVE_ENGAGE_INT3</td><td>4.81</td><td>.887</td></tr></table>

## 5.1 Measurement Model Evaluation

<table><caption>Table 3. Measurement of Validity and Reliability</caption><thead><tr><th></th><th>CR</th><th>AVE</th><th>GE</th><th>PUGame</th><th>ATT_COURSE</th><th>COG_ENG_INT</th></tr></thead><tbody><tr><td>Game Elements</td><td>0.836</td><td>0.506</td><td><strong>0.711</strong></td><td></td><td></td><td></td></tr><tr><td>Perceived Usefulness of Games</td><td>0.865</td><td>0.616</td><td>.508</td><td><strong>0.785</strong></td><td></td><td></td></tr><tr><td>Attitude towards the Course</td><td>0.888</td><td>0.726</td><td>.438</td><td>.563</td><td><strong>0.852</strong></td><td></td></tr><tr><td>Cognitive Engagement Intention</td><td>0.936</td><td>0.831</td><td>.469</td><td>.494</td><td>.706</td><td><strong>0.912</strong></td></tr></tbody></table>

The composite reliability (CR) and the average variance extracted (AVE) are measured to check the internal consistency of constructs. The CR values of game elements, perceived usefulness of games, attitude towards the course, and cognitive engagement intention, greater than .60 [15, 47] ranging from 0.836 to 0.936, show a high level of convergent validity. In addition, all AVE values, ranging from 0.506 to 0.831, are above the recommended threshold of 0.5 [47], as shown in Table 3. Therefore, all factors present the good internal consistency. Discriminant validity is evaluated by comparing the constructs' correlations with the square root of the AVEs, showing in the bold fonts in Table 3. The correlations among all latent variables are less than 1 [47]. The square root of AVEs is greater than the correlations, indicating the adequate discriminant va-
lidity [20]. The discriminant validity is also examined using Confirmatory Factor Anal-
ysis. The CFA model with 4 latent variables and 15 measurement items fits the data
quite well. The goodness-of-fit statistics are as follows: χ2 (83) = 124.151, p = .002,
CMIN/DF = 1.496, RMR = .043, GFI = .927, NFI = .926, TLI = .967, IFI = .974, CFI
= .974, PNFI = .732, PCFI = .770, NCP = 41.151, RMSEA = .057, AIC = 198.151,
ECVI = .953, according to the guidelines for determining the model fit [48-50]. Thus,
the test reveals the sufficient discriminant validity.

5.2 Structural Model Evaluation

<figure><img src="image_3.png" /><figcaption>Fig. 2. Parameter Estimated Results for the Model</figcaption></figure>

The overall fit of the structural model is satisfactory, according to the guidelines for
evaluating the SEM model fit [48-50]. The proposed model presents a good structure
that underlies the collected data (χ2 (83) = 124.151, p = .002, CMIN/DF = 1.496, RMR
= .043, GFI = .927, NFI = .926, TLI = .967, IFI = .974, CFI = .974, PNFI = .732, PCFI
= .770, NCP = 41.151, RMSEA = .049, AIC = 198.151, ECVI = .953). The χ2 is sig-
nificant. However, CMIN/DF is less than 2, indicating an acceptable model fit. Figure
2 shows the relationships between game elements, perceived usefulness of games, atti-
tude towards the course, and cognitive engagement intention. It is found that game el-
ements are significant predictors of perceived game usefulness (β = .51, p < .001), atti-
tude towards the course (β = .21, p < .05), and cognitive engagement intention (β = .17,
p < .05), supporting the hypotheses H1 – H3. The influence of perceived game useful-
ness on attitude towards the course (β = .46, p < .001) is also significant, confirming
the hypothesis H4. Nevertheless, there is no significant direct relationship between per-
ceived game usefulness and the cognitive engagement intention (β = .07, p < .371), so
the hypothesis H5 is rejected. Attitude towards the course is a significant predictor of
cognitive engagement intention (β = .59, p < .001), in support of the hypothesis H6.
The structural model has a considerable explanatory power, with the R² values of 25.8% for perceived game usefulness, 35% for attitude towards the course, and 53% for cognitive engagement intention. Five of six hypotheses are conformed to the previous studies. However, the rejection of hypothesis H5 is explained by the insignificance of the proposed hypothesis of Ortiz-Rojas, Chiluiza and Valcke [14]: students involved in a gamified course will attain higher learning gains as compared to students in a control condition. In addition, enjoyment received from the game has no significant impact on the cognitive learning gain [19]. Enjoyment, excitement, perceived competence, and perceived improvement in the gamified group are also not significantly different from the non-gamified group [13].

# 6 Theoretical and Managerial Implications

For theoretical implications, this study confirms the validity of measures in the literature and the developed scales. These measurement items such as perceived usefulness and cognitive engagement intention can be adapted in the future research to expand the view of those variables. This study also presents an experimental design using a simple process to implement gamification in classrooms, using student-led gamification. The experimental design can be adapted into other specific contexts of gamified courses to make the implementation of gamification easier. The model, built on the past behavioral research, can be extended with other constructs, such as behavioral and emotional engagement intention and other drivers of these factors. Moreover, this study explicitly links the use of games with the expected outcomes, showing the effectiveness of gamification in education.

For managerial implications, teachers could easily gamify their classrooms using student-led gamification activities. Using student-led gamification is believed to be better understanding the players (peers) more than the use of games designed by teachers. Student-led activities have also normally incorporated various game elements. In addition, the objectives of each activity would be distinctly declared, if teachers clearly define the assessment method in details. For example, after the game ends in each round, game participants evaluate the game in 5 aspects: game enjoyment, content coverage, content reliability, the harmony of the game conductor group, and overall satisfaction. Findings from the structural model evaluation point that game elements directly affect perceived usefulness, attitude towards the course, and cognitive engagement intention, particularly the perceived game usefulness. Game elements are direct and indirect predictors of attitude towards the course and cognitive engagement intention. Thus, Awards or prizes for winners should be various and creative. Gamified activities should have specific rules such as special items, testing one's luck, scarcity of resources, random timeout, and unpredictable alterations. Leaderboard or player's status should be displayed using colorful graphics or audio signals such as music or announcements from game leaders. Games should contain a form of communication or interaction between players such as exchanging items. Players should be able to personalize their profiles such as selecting and naming their avatars. Perceived usefulness of games directly influences student's attitude towards the course. Therefore, teachers and game
conductors should emphasize the usefulness of game in terms of improving students’ understanding, increasing creativity, persuading them to pay more attention to the topics, and promoting their problem-solving skills, to promote students’ positive attitudes toward the course and the gamified topics. Lastly, the attitude towards the course has a strong impact on the cognitive engagement intention of students. Students will have the positive attitudes toward the course if they feel good about the course; they think the course is interesting; or they like the course. The good attitudes toward the course will later lead students to try to gain more understanding about the lessons, to learn more about the lessons, and to study the lessons intensively. Positive attitudes toward the course could be shaped through students’ comprehension of game usefulness and the suitable combination of game elements respectively.

# 7 Conclusion, Limitations, and Future Research

Engaging students in learning activities is important for education in the 21st century. Incorporating games into in-class activities is one approach to increase students’ engagement and participation. However, the effectiveness of gamification on the educational outcomes such as attitude towards the course and cognitive engagement intention has not yet been explored. Therefore, this work aims to examine the effects of game elements on the perceived usefulness of games, attitude towards the course, and cognitive engagement intention, to investigate the relationships among these factors, to rank their importance, and to guide an easy but effective method for instructors to conduct gamification in the classroom. The results show students’ perceived understanding, problem-solving skills, creativity, and their interests in the topics after conducting games are quite high. Game elements are predictors of perceived game usefulness, attitude towards the course, and cognitive engagement intention respectively. Perceived game usefulness significantly affects the attitude towards the course, but insignificantly influences cognitive engagement intention. Attitude towards the course is vital to increase students’ cognitive engagement intention.

The limitation of this study includes the small sample size. Although the experimental studies are conducted in five rounds, there are still the relatively small samples. In the future, more samples should be added to better generalize the results. Comparative studies should be conducted, between the preliminary course and the advanced course or between IS/IT course and other subjects. Comparing the effectiveness of different game elements, which will be implemented in student-led gamification, is also an area for future study. A comparison between student-led gamified activities versus teacher-led gamified activities should be drawn. Comparing between gamification and other active learning methods such as flipping classroom should be done in the future.

# References

1. da Rocha Seixas, L., Gomes, A.S., de Melo Filho, I.J.: Effectiveness of gamification in the engagement of students. Computers in Human Behavior 58, 48-63 (2016)
2. Mahmud, Z., Weber, P.J., Moening, J.P.: Gamification of Engineering Courses. In: ASEE Annual conference and exposition, conference proceedings. (2017)

3. Thiebes, S., Lins, S., Basten, D.: Gamifying information systems-a synthesis of gamification mechanics and dynamics. In: European Conference on Information Systems (ECIS2014). (2014)

4. Cheong, C., Filippou, J., Cheong, F.: Towards the gamification of learning: Investigating student perceptions of game elements. Journal of Information Systems Education 25, 233 (2014)

5. Azmi, S., Iahad, N.A., Ahmad, N.: Gamification in online collaborative learning for programming courses: a literature review. ARPN Journal of Engineering and Applied Sciences 10, 1-3 (2015)

6. Combefis, S., Bersnevicius, G., Dagiene, V.: Learning Programming through Games and Contests: Overview, Characterisation and Discussion. INTERNATIONAL OLYMPIAD IN INFORMATICS Vol. 10: 39-60. Vilnius: Vilniaus universiteto leidykla. ISSN 1822 7732, (2016)

7. Yang, Y., Asaad, Y., Dwivedi, Y.: Examining the impact of gamification on intention of engagement and brand attitude in the marketing context. Computers in Human Behavior (2017)

8. Paisley, V.: Gamification of tertiary courses: An exploratory study of learning and engagement. In: ASCIIITE-Australian Society for Computers in Learning in Tertiary Education Annual Conference, pp. 671-675. Australasian Society for Computers in Learning in Tertiary Education, (2013)

9. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: Potential of the concept, state of the evidence. Review of educational research 74, 59-109 (2004)

10.Osipov, I.V., Nikulchev, E., Volinsky, A.A., Prasikova, A.Y.: Study of gamification effectiveness in online e-learning systems. International Journal of advanced computer science and applications 6, 71-77 (2015)

11.Hamari, J., Koivisto, J., Sarsa, H.: Does Gamification Work?--A Literature Review of Empirical Studies on Gamification. In: the 47th Hawaii International Conference on System Sciences (HICSS), pp. 3025-3034. IEEE, (2014)

12.Qu, W.-Q., Zhao, Y.-F., Wang, M., Liu, B.-Q.: Research on teaching gamification of software engineering. In: Computer Science & Education (ICCSE), 2014 9th International Conference on, pp. 855-860. IEEE, (2014)

13.Silpasuwanchai, C., Ma, X., Shigemasu, H., Ren, X.: Developing a comprehensive engagement framework of gamification for reflective learning. In: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, pp. 459-472. ACM, (2016)

14.Ortiz-Rojas, M., Chiluiza, K., Valcke, M.: Gamification in Computer Programming: Effects on Learning, Engagement, Self-Efficacy and Intrinsic Motivation. In: European Conference on Games Based Learning, pp. 507-514. Academic Conferences International Limited, (2017)

15.Galbis Córdova, A., Martí Parreño, J., Currás Pérez, R.: Higher education students' attitude towards the use of gamification for competencies development. (2017)

16.Silpasuwanchai, C.: A Study of Game Engagement through Gamification and Full-body Games. Kochi University of Technology (2015)

17.Pintrich, P.R., Schrauben, B.: Students' motivational beliefs and their cognitive engagement in classroom academic tasks. Student perceptions in the classroom 7, 149-183 (1992)

18.Mitchell, N., Danino, N., May, L.: Motivation and manipulation: A gamification approach to influencing undergraduate attitudes in computing. In: European Conference on Games Based Learning, pp. 394. Academic Conferences International Limited, (2013)
19. Iten, N., Petko, D.: Learning with serious games: Is fun playing the game a predictor of learning success? British Journal of Educational Technology 47, 151-163 (2016)

20. Putz, L.-M., Treiblmaier, H.: Gamified Workshops as Drivers for Attitudinal and Behavioral Shifts toward Sustainable Business Practices: The Role of Enjoyment, Curiosity and External Regulation. In: Proceedings of the 51st Hawaii International Conference on System Sciences. (2018)

21. Tomaselli, F., Sanchez, O., Brown, S.: How to Engage Users through Gamification: The Prevalent Effects of Playing and Mastering over Competing. In: International Conference on Information Systems (ICIS 2015), pp. 1-16. (2015)

22. Rouibah, K.: Understanding student drivers and obstacles toward MIS major from the perspective of an arab country: The case of Kuwait. Issues in Information Systems 13, 58-71 (2012)

23. Vos, N., Van Der Meijden, H., Denessen, E.: Effects of constructing versus playing an educational game on student motivation and deep learning strategy use. Computers & Education 56, 127-137 (2011)

24. Surendeleg, G., Murwa, V., Yun, H.-K., Kim, Y.S.: The role of gamification in education a literature review. Contemporary Engineering Sciences 7, 1609-1616 (2014)

25. Poole, S.M., Kemp, E., Patterson, L., Williams, K.: Get your head in the game: Using gamification in business education to connect with Generation Y. (2014)

26. Tan, M., Hew, K.F.: Incorporating meaningful gamification in a blended learning research methods class: Examining student learning, engagement, and affective outcomes. Australasian Journal of Educational Technology 32, (2016)

27. Fotaris, P., Mastoras, T., Leinfellner, R., Rosunally, Y.: Who wants to be a Pythonista? Using Gamification to Teach Computer Programming. In: 7th International Conference on Education and New Learning Technologies, pp. 2611-2619. (2015)

28. Veltsos, J.R.: Gamification in the Business Communication Course. Business and Professional Communication Quarterly 80, 194-216 (2017)

29. Chiu, M.M., Pong, S.-l., Mori, I., Chow, B.W.-Y.: Immigrant students' emotional and cognitive engagement at school: A multilevel analysis of students in 41 countries. Journal of youth and adolescence 41, 1409-1425 (2012)

30. Barata, G., Gama, S., Jorge, J., Gonçalves, D.: Engaging engineering students with gamification. In: the 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), pp. 1-8. IEEE, (2013)

31. Smith, T.: Gamified Modules for an Introductory Statistics Course and Their Impact on Attitudes and Learning. Simulation & Gaming 48, 832-854 (2017)

32. Gardner, C., Amoroso, D.L.: Development of an instrument to measure the acceptance of internet technology by consumers. In: System Sciences, 2004. Proceedings of the 37th Annual Hawaii International Conference on, pp. 10 pp. IEEE, (2004)

33. Abdalla, I.: Evaluating effectiveness of e-blackboard system using TAM framework: A structural analysis approach. Aace Journal 15, 279-287 (2007)

34. Tao, J., Li, Z.: A Case Study on Computerized Take-Home Testing: Benefits and Pitfalls. International Journal of Technology in Teaching & Learning 8, 33-43 (2012)

35. Aleksic-Maslac, K., Sinkovic, B., Vranesic, P.: Influence of gamification on student engagement in education. International Journal of Education and Learning Systems 2, 76-82 (2017)
36. Kanagasingam, G.: Increasing the Learning Motivations Using Gamification. Journal of Administrative Management, Education and Training 13, 192-198 (2017)

37. Galluch, P., Thatcher, J.B.: Maladaptive vs. adaptive use of internet applications in the classroom: a test of competing models. In: Proceedings of the 2007 southern association for information systems conference, pp. 9-10. (2007)

38. Scott, J.E., Walczak, S.: Cognitive engagement with a multimedia ERP training tool: Assessing computer self-efficacy and technology acceptance. Information & Management 46, 221-232 (2009)

39. Lin, W.-S., Wang, C.-H.: Antecedences to continued intentions of adopting e-learning system in blended learning instruction: A contingency framework based on models of information system success and task-technology fit. Computers & Education 58, 88-99 (2012)

40. Liaw, S.-S., Huang, H.-M.: Information retrieval from the World Wide Web: a user-focused approach based on individual experience with search engines. Computers in human behavior 22, 501-517 (2006)

41. Hsu, S.H., Chang, J.-W., Lee, C.-C.: Designing attractive gamification features for collaborative storytelling websites. Cyberpsychology, Behavior, and Social Networking 16, 428-435 (2013)

42. Robinson, D., Bellotti, V.: A preliminary taxonomy of gamification elements for varying anticipated commitment. In: ACM CHI 2013 Workshop on Designing Gamification: Creating Gameful and Playful Experiences. (2013)

43. Cheong, C., Filippou, J., Cheong, F.: Understanding Student Perceptions of Game Elements to Develop Gamified Systems for Learning. In: PACIS, pp. 202. (2013)

44. Kenny, G., Lyons, R., Lynn, T.: Don't Make the Player, Make the Game: Exploring the Potential of Gamification in IS Education. In: Americas Conference on Information Systems. (2017)

45. Hamari, J., Koivisto, J.: Social Motivations To Use Gamification: An Empirical Study Of Gamifying Exercise. In: ECIS, pp. 105. (2013)

46. Ibanez, M., Di Serio, A., Delgado Kloos, C.: Gamification for Engaging Computer Science Students in Learning Activities: A Case Study. IEEE Transactions on Learning Technologies 7, 291-301 (2014)

47. Bagozzi, R.P., Yi, Y.: On the evaluation of structural equation models. Journal of the academy of marketing science 16, 74-94 (1988)

48. Schreiber, J.B., Nora, A., Stage, F.K., Barlow, E.A., King, J.: Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of educational research 99, 323-338 (2006)

49. Hooper, D., Coughlan, J., Mullen, M.: Structural equation modelling: Guidelines for determining model fit. Articles 2 (2008)

50. Schermelleh-Engel, K., Moosbrugger, H., Müller, H.: Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of psychological research online 8, 23-74 (2003)