# Investigating and Comparing the Effects on Learning Achievement and Motivation for Gamification and Game-Based Learning: A Quantitative Study Employing Kahoot

Authors: Qi Zhang; Zhonggen Yu
Journal: Education Research International
 (2022)

---

## Abstract

This study is aimed at investigating and comparing the effects on learning achievement and motivation for two game-related pedagogies: gamification and game-based learning. Gamification was the process involving separable and flexible game elements, while game-based learning was the pedagogical procedure based on the inseparable serious games. Thus, gamification and game-based learning were hypothesized to have different effects on learning achievement and motivation. We implemented College English Test-6 (CET-6) and Motivated Strategies for Learning Questionnaire (MSLQ) to quantitatively assess learning achievement and motivation. ANCOVA reported that the positive effects on learning achievement reflected by CET-6 posttest scores were more significant for gamification (M=79.301, SD=1.258) than game-based learning (M=77.473, SD=1.262). ANOVA revealed that the positive effects on motivation reflected by the motivation-related subscales, i.e., self-efficacy for learning and performance, extrinsic goal orientation, intrinsic goal orientation, and control of learning beliefs, were more significant for gamification than game-based learning. The main conclusion was that gamification exerted more significantly positive effects on learning achievement and motivation than game-based learning. However, high dependence on immersion might influence the stability for the effects of gamification.


---

## Body

## 1. Introduction

Game refers to a series of interesting and meaningful choices allowing players to achieve precise and compelling goals [1]. Technological upgrading and development allowed educators to implement game or game-related designs in pedagogical practices [2–6]. Games in education contexts stand for the designs created not for pure entertainment but for the serious intentions, e.g., learning, training, or health care [7]. Along with the implementations of games or game-related designs in pedagogical practices, educators accordingly proposed numerous game-related concepts and pedagogies, e.g., the digital game-based learning (DGBL) model [8], game-based flipped learning [9, 10], and serious educational games (SEG) model [11]. These concepts and pedagogies were proposed to guarantee learners’ enjoyment and promising instruction effectiveness [4, 12, 13].Gamification and game-based learning are among the proposed game-related concepts and pedagogies. These seemingly similar terms were usually regarded as interchangeable concepts [14]. However, strictly speaking, gamification and game-based learning are different terms with distinct features [5, 13]. For instance, gamification and game-based learning have different definitions. Gamification refers to the process of utilizing digital game mechanics in originally nongaming contexts to engage learners, enhance learning, and solve problems [4, 6, 13, 15]. In contrast, game-based learning refers to the pedagogical process where educators use games or related designs for educational purposes rather than entertainment [5, 13, 16].For that reason, we would regard these distinct concepts as noninterchangeable terms in this study. Thus, this study is aimed at investigating hypothesized positive effects of game-related pedagogies by comparing the efficiencies for gamification and game-based learning. Based on the distinctions between these game-related pedagogies at the theoretical level, we could accordingly identify how differently these pedagogies function at the empirical level. Considering the dimensions to quantitatively measure the efficiencies of gamification and game-based learning, learning achievement is a typical criterion to evaluate the efficiency of particular pedagogies [4]. Besides, motivation could be another criterion because game-related pedagogies could promote learners’ psychological changes by cultivating their motivation [4, 12]. Therefore, this study compared the efficiencies for gamification and game-based learning in terms of learning achievement and motivation.Considering the potential implications of this study, this study emphasizes the differences between gamification and game-based learning in terms of theoretical distinctions and their potentially different efficiencies in learning achievement and motivation. Although implementing games in pedagogical practices is an innovative approach with hypothesized motivating effects [4, 12], educators and researchers still need to differentiate between gamification and game-based learning, consider potential influences, and select proper pedagogies based on practical needs. The findings could also reflect how differently gamification and game-based learning function, based on which educators could accordingly refine game-related models or pedagogies to maximize their efficiency.Literature Review discussed the distinctions between gamification and game-based learning (Section2.1) and described the target dimensions for quantitatively measuring the efficiencies of gamification and game-based learning (Section 2.2), based on which we raised the research hypotheses. Methodology described the participants involved in this empirical study (Section 3.1), the research instruments for quantitative measurements (Section 3.2), the research procedures (Section 3.3), and analytical approaches based on the collected materials (Section 3.4). Results reported the findings to test the hypotheses. Discussion focused on the comparison for the results and those of the past studies, accompanied with the speculation on stability of the effects for the game-related pedagogies. The Conclusion section provided brief summaries of findings to test the hypotheses (Section 6.1) and offered insights for future research based on the limitations of this study (Section 6.2).


## 2. Literature Review

This section provides theoretical foundations of the research hypotheses.


### 2.1. The Distinction between Gamification and Game-Based Learning

The term “gamification” first appeared in late 2010 [17]. As this term emerged, numerous terms appeared and denoted game-related concepts [18]. The distinction between gamification and game-based learning is consistent with the distinction between gamification and game-based learning.


#### 2.1.1. Gamification and Serious Game Based on “Game versus Play” and “Whole and Elements”

Deterding et al. [17] proposed two dimensions: “game versus play” and “whole versus elements” to differentiate gamification from other related concepts (see Figure 1). “Game versus play” reflects whether the target game-related product involves rule-bounded and outcome-related elements or just consists of the entertainment aspect [17, 18]. “Gaming” stands for rule-based and goal-oriented playful activities based on explicit rule systems and outcomes, while “playing” refers to free-form, nonrule-based, and expressive actions [19]. “Whole versus parts” demonstrates the extent to which the target circumstance is employing game elements [17, 18]. “Whole” reflects the application of entire games, while “elements” reflects using game elements rather than entire games [17].Figure 1
Game-related concepts in the dimensions of whole versus parts and gaming versus playing [17].The dimensions of “game versus playing” and “whole and elements” suggest both gamification and serious game approximate to “game.” Gamification and serious game emphasize rule-based goal-oriented designs that encourage players to progress by completing tasks or surpassing others [17, 18]. Besides, gamification reflects using separable game-related elements, whereas serious games require complete and inseparable games with an education or learning background [17, 18].Additionally, gamification and serious game differ in the way to offer engagement experiences. Both gamification and serious game necessity engagement experiences promote or constrain specific behaviors and reach goals [20]. Gamification usually offers engagement in a relatively incorporated manner, while serious game usually provides implicit engagement in a relatively holistic manner [20]. Incorporated and holistic manners are consistent with the “element” feature for gamification and the “whole” feature for serious game, respectively.Thus, the distinction between gamification and serious game is the separability of the game designs. Gamification means the implementations of separable game elements rather than complete games, while serious game requires inseparable educational games not for entertainment. Gamification is a process of using game elements rather than specific games, while serious game is the inseparable game.


#### 2.1.2. Gamification versus Game-Based Learning Based on Gamification versus Serious Game

The distinction between gamification and serious game lies in separability of the target game-related designs. That distinction is consistent with the distinction between gamification and game-based learning.Gamification stands for the application of digital game mechanics in naturally nongaming contexts to engage learners, enhance learning, and solve problems [4, 6, 13, 15, 17]. In the etymological aspect, the Latin root “facere” in “gamification” indicates the action of “making the game” [21]. Gamifying contexts does not necessarily require complete games or game designs [17, 18]. Gamification emphasizes the dominant role of game elements in pedagogical practices and the immersive experiences [4, 5, 13].In contrast, game-based learning refers to the application of complete games to deliver learning rather than entertainment [5, 16]. Notably, serious game is inseparable and consistent with educational or pedagogical purposes [11, 17, 18]. Game-based learning means the implementation of serious games [5, 13]. For game-based learning, inseparable serious games are the essential auxiliary instruments to attract learners to achieve learning based on interactive and enjoyable activities [5, 13, 22].Overall, gamification and game-based learning are distinct game-related pedagogies. The major differences between gamification and game-based learning lie in the separability of the game-related designs and how the educators emphasize game-related components in pedagogical practices. Gamification is the process involving separable and flexible game elements as the dominant components in pedagogical practices, whereas game-based learning reflects the pedagogical procedure based on the inseparable serious games as the essential auxiliary components.


### 2.2. Learning Achievement and Motivation as the Dimensions to Compare the Efficiencies for Gamification and Game-Based Learning

Since gamification and game-based learning are different game-related pedagogies, we would investigate and compare their efficiencies in this empirical study. We chose learning achievement and motivation as the target dimensions for quantitative measurements of efficiencies.Learning achievement is the common measurement assessing as the dimension assessing the efficiency of specific pedagogies [4]. Past studies, e.g., [9, 23–29]) investigated and found the positive impacts of game-related pedagogies on learning achievement based on the vivid and motivating presentations of teaching contents. Specifically speaking, game-related pedagogies were found conducive to independent higher-order thinking skills and test performance [2, 15, 30–36].Apart from learning achievement, psychological and behavioral changes are other important references reflecting the efficiencies of game-related pedagogies [4]. Since game-related pedagogies also emphasize learners’ psychological changes by cultivating their motivation [4, 12, 22], motivation is another criterion to evaluate the efficiency of game-related pedagogies. Recent studies, e.g., [2, 15, 29, 37–41], have demonstrated the significant effects of game-related pedagogies on motivation.Since past studies revealed the positive effects of game-related pedagogies on learning achievement and motivation, we chose learning achievement and motivation as the dimensions for quantitative measurements of efficiencies. Since gamification and game-based learning are different game-related pedagogies, we would assume that gamification and game-based learning show different efficiencies in terms of learning achievement and motivation and thus raise the following hypotheses.(i)
H1: gamification and game-based learning exert differently positive effects on learning achievement(ii)
H2: gamification and game-based learning exert differently positive effects on motivation(iii)
To test these hypotheses, we designed the empirical procedures, collected the materials, and performed the statistical processes


## 2.1. The Distinction between Gamification and Game-Based Learning

The term “gamification” first appeared in late 2010 [17]. As this term emerged, numerous terms appeared and denoted game-related concepts [18]. The distinction between gamification and game-based learning is consistent with the distinction between gamification and game-based learning.


### 2.1.1. Gamification and Serious Game Based on “Game versus Play” and “Whole and Elements”

Deterding et al. [17] proposed two dimensions: “game versus play” and “whole versus elements” to differentiate gamification from other related concepts (see Figure 1). “Game versus play” reflects whether the target game-related product involves rule-bounded and outcome-related elements or just consists of the entertainment aspect [17, 18]. “Gaming” stands for rule-based and goal-oriented playful activities based on explicit rule systems and outcomes, while “playing” refers to free-form, nonrule-based, and expressive actions [19]. “Whole versus parts” demonstrates the extent to which the target circumstance is employing game elements [17, 18]. “Whole” reflects the application of entire games, while “elements” reflects using game elements rather than entire games [17].Figure 1
Game-related concepts in the dimensions of whole versus parts and gaming versus playing [17].The dimensions of “game versus playing” and “whole and elements” suggest both gamification and serious game approximate to “game.” Gamification and serious game emphasize rule-based goal-oriented designs that encourage players to progress by completing tasks or surpassing others [17, 18]. Besides, gamification reflects using separable game-related elements, whereas serious games require complete and inseparable games with an education or learning background [17, 18].Additionally, gamification and serious game differ in the way to offer engagement experiences. Both gamification and serious game necessity engagement experiences promote or constrain specific behaviors and reach goals [20]. Gamification usually offers engagement in a relatively incorporated manner, while serious game usually provides implicit engagement in a relatively holistic manner [20]. Incorporated and holistic manners are consistent with the “element” feature for gamification and the “whole” feature for serious game, respectively.Thus, the distinction between gamification and serious game is the separability of the game designs. Gamification means the implementations of separable game elements rather than complete games, while serious game requires inseparable educational games not for entertainment. Gamification is a process of using game elements rather than specific games, while serious game is the inseparable game.


### 2.1.2. Gamification versus Game-Based Learning Based on Gamification versus Serious Game

The distinction between gamification and serious game lies in separability of the target game-related designs. That distinction is consistent with the distinction between gamification and game-based learning.Gamification stands for the application of digital game mechanics in naturally nongaming contexts to engage learners, enhance learning, and solve problems [4, 6, 13, 15, 17]. In the etymological aspect, the Latin root “facere” in “gamification” indicates the action of “making the game” [21]. Gamifying contexts does not necessarily require complete games or game designs [17, 18]. Gamification emphasizes the dominant role of game elements in pedagogical practices and the immersive experiences [4, 5, 13].In contrast, game-based learning refers to the application of complete games to deliver learning rather than entertainment [5, 16]. Notably, serious game is inseparable and consistent with educational or pedagogical purposes [11, 17, 18]. Game-based learning means the implementation of serious games [5, 13]. For game-based learning, inseparable serious games are the essential auxiliary instruments to attract learners to achieve learning based on interactive and enjoyable activities [5, 13, 22].Overall, gamification and game-based learning are distinct game-related pedagogies. The major differences between gamification and game-based learning lie in the separability of the game-related designs and how the educators emphasize game-related components in pedagogical practices. Gamification is the process involving separable and flexible game elements as the dominant components in pedagogical practices, whereas game-based learning reflects the pedagogical procedure based on the inseparable serious games as the essential auxiliary components.


## 2.1.1. Gamification and Serious Game Based on “Game versus Play” and “Whole and Elements”

Deterding et al. [17] proposed two dimensions: “game versus play” and “whole versus elements” to differentiate gamification from other related concepts (see Figure 1). “Game versus play” reflects whether the target game-related product involves rule-bounded and outcome-related elements or just consists of the entertainment aspect [17, 18]. “Gaming” stands for rule-based and goal-oriented playful activities based on explicit rule systems and outcomes, while “playing” refers to free-form, nonrule-based, and expressive actions [19]. “Whole versus parts” demonstrates the extent to which the target circumstance is employing game elements [17, 18]. “Whole” reflects the application of entire games, while “elements” reflects using game elements rather than entire games [17].Figure 1
Game-related concepts in the dimensions of whole versus parts and gaming versus playing [17].The dimensions of “game versus playing” and “whole and elements” suggest both gamification and serious game approximate to “game.” Gamification and serious game emphasize rule-based goal-oriented designs that encourage players to progress by completing tasks or surpassing others [17, 18]. Besides, gamification reflects using separable game-related elements, whereas serious games require complete and inseparable games with an education or learning background [17, 18].Additionally, gamification and serious game differ in the way to offer engagement experiences. Both gamification and serious game necessity engagement experiences promote or constrain specific behaviors and reach goals [20]. Gamification usually offers engagement in a relatively incorporated manner, while serious game usually provides implicit engagement in a relatively holistic manner [20]. Incorporated and holistic manners are consistent with the “element” feature for gamification and the “whole” feature for serious game, respectively.Thus, the distinction between gamification and serious game is the separability of the game designs. Gamification means the implementations of separable game elements rather than complete games, while serious game requires inseparable educational games not for entertainment. Gamification is a process of using game elements rather than specific games, while serious game is the inseparable game.


## 2.1.2. Gamification versus Game-Based Learning Based on Gamification versus Serious Game

The distinction between gamification and serious game lies in separability of the target game-related designs. That distinction is consistent with the distinction between gamification and game-based learning.Gamification stands for the application of digital game mechanics in naturally nongaming contexts to engage learners, enhance learning, and solve problems [4, 6, 13, 15, 17]. In the etymological aspect, the Latin root “facere” in “gamification” indicates the action of “making the game” [21]. Gamifying contexts does not necessarily require complete games or game designs [17, 18]. Gamification emphasizes the dominant role of game elements in pedagogical practices and the immersive experiences [4, 5, 13].In contrast, game-based learning refers to the application of complete games to deliver learning rather than entertainment [5, 16]. Notably, serious game is inseparable and consistent with educational or pedagogical purposes [11, 17, 18]. Game-based learning means the implementation of serious games [5, 13]. For game-based learning, inseparable serious games are the essential auxiliary instruments to attract learners to achieve learning based on interactive and enjoyable activities [5, 13, 22].Overall, gamification and game-based learning are distinct game-related pedagogies. The major differences between gamification and game-based learning lie in the separability of the game-related designs and how the educators emphasize game-related components in pedagogical practices. Gamification is the process involving separable and flexible game elements as the dominant components in pedagogical practices, whereas game-based learning reflects the pedagogical procedure based on the inseparable serious games as the essential auxiliary components.


## 2.2. Learning Achievement and Motivation as the Dimensions to Compare the Efficiencies for Gamification and Game-Based Learning

Since gamification and game-based learning are different game-related pedagogies, we would investigate and compare their efficiencies in this empirical study. We chose learning achievement and motivation as the target dimensions for quantitative measurements of efficiencies.Learning achievement is the common measurement assessing as the dimension assessing the efficiency of specific pedagogies [4]. Past studies, e.g., [9, 23–29]) investigated and found the positive impacts of game-related pedagogies on learning achievement based on the vivid and motivating presentations of teaching contents. Specifically speaking, game-related pedagogies were found conducive to independent higher-order thinking skills and test performance [2, 15, 30–36].Apart from learning achievement, psychological and behavioral changes are other important references reflecting the efficiencies of game-related pedagogies [4]. Since game-related pedagogies also emphasize learners’ psychological changes by cultivating their motivation [4, 12, 22], motivation is another criterion to evaluate the efficiency of game-related pedagogies. Recent studies, e.g., [2, 15, 29, 37–41], have demonstrated the significant effects of game-related pedagogies on motivation.Since past studies revealed the positive effects of game-related pedagogies on learning achievement and motivation, we chose learning achievement and motivation as the dimensions for quantitative measurements of efficiencies. Since gamification and game-based learning are different game-related pedagogies, we would assume that gamification and game-based learning show different efficiencies in terms of learning achievement and motivation and thus raise the following hypotheses.(i)
H1: gamification and game-based learning exert differently positive effects on learning achievement(ii)
H2: gamification and game-based learning exert differently positive effects on motivation(iii)
To test these hypotheses, we designed the empirical procedures, collected the materials, and performed the statistical processes


## 3. Methodology

This section describes participants, instruments, procedures, and statistical approaches of this empirical study.


### 3.1. Research Participants

This study involved 120 participants (86 female students and 34 male students) who were seniors majoring in fine arts from universities. The participants attended the courses of English for general purposes to enhance their English proficiency.We randomly divided these participants into three groups similar in scales and distributions of female and male participants (see Figure2). The first experimental group involved 40 participants, including 29 female students and 11 male students. The second experimental group involved 40 participants, including 30 female students and ten male students. The control group involved 40 participants, including 27 female students and 13 male students.Figure 2
The experiment procedures assessing the impacts of game-related pedagogies on learning achievement and motivation.Participants in the first experimental group attended the English courses employing gamification. Participants in the second experimental group attended the courses employing game-based learning. Participants in the control group attended the courses adopting traditional didactic pedagogy without any game-related designs. Setting a control group could demonstrate whether game-related pedagogies have significant positive impacts on learning achievement and motivation. Setting two experimental groups enabled us to compare the quantitative assessments of learning achievement and motivation for gamification and game-based learning.


### 3.2. Research Instruments

Kahoot! was the platform to perform gamification and game-based learning. We employed the English proficiency test and motivation-related questionnaire as the instruments for quantitative assessments of learning achievement and motivation, respectively.


#### 3.2.1. Kahoot! as the Platform for Gamification and Game-Based Learning

We utilized Kahoot! (https://kahoot.com) to perform game-related pedagogies. Initially developed in 2012 and published in 2013, Kahoot! is a game-based student response system (GSRS) transforming the classroom into a game show [42–44].Past studies revealed the positive effects of Kahoot!. Most learners stated that Kahoot! could enhance their motivation, enjoyment, excitement, engagement, learning experiences, and learning efficiency [44]. Besides, Kahoot! provided a user-friendly atmosphere, attractive images, and dynamic music that could stimulate learners’ interest in learning [43]. Kahoot! could also positively influence academic performance and stimulate active attitudes to learning [25, 26].We employed Kahoot! in two ways to perform gamification and game-based learning. Since gamification requires separable game elements rather than a complete game, the first experimental group has used the “Host Live” mode involving the elements of game shows: points, leaderboards, count-down tick, and podium for immersive playful activities. Since game-based learning requires complete game(s) that play an essential auxiliary role in pedagogical practices, the second experimental group has employed the “self-paced kahoots” mode as an independent and complete game design involving cycles of learning, reciting, and practicing.


#### 3.2.2. College English-6 (CET-6) for Quantitatively Assessing Learning Achievement

We employed adapted versions of College English Test-6 (CET-6) to quantitatively assess participants’ learning achievement in English proficiency. The authoritative roles of College English Test-4 (CET-4) and College English-6 (CET-6) in China contribute to the relatively reliable and persuasive results about English proficiency from CET-6.We employed adapted CET-6 in the forms of pretest and posttest to assess participants’ English foundation before attending the courses and English proficiency after attending the courses, respectively. The total score was 100 for both pretest and posttest. Both pretest and posttest contained ten cloze questions, ten paragraph matching questions, ten multiple-choice questions, one Chinese to English translation task, and one writing task from CET-6.


#### 3.2.3. Motivated Strategies for Learning Questionnaire (MSLQ) for Quantitatively Assessing Motivation

We implemented the adapted Motivated Strategies for Learning Questionnaire (MSLQ) in the study of Pintrich, Smith, García, and McKeachie [45] (see Table 1). The adapted questionnaire consisted of 18 questions scored with a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The 18 questions belonged to four subscales: extrinsic goal orientation (Q1.1-Q1.4), intrinsic goal orientation (Q2.1-Q2.4), control of learning beliefs (Q3.1-3.4), and self-efficacy for learning and performance (Q4.1-Q4.6).Table 1
The adapted version of Motivated Strategies for Learning Questionnaire (MSLQ) in the study of Pintrich, Smith, García, and McKeachie [45].
Motivation-related subscalesNo.Questions scored with a 5-point Likert scaleIntrinsic goal orientationQ1.1In a class like this, I prefer course material that really changes me so I can learn new things.Q1.2In a class like this, I prefer course material that arouses my curiosity, even if it is difficult to learn.Q1.3The most satisfying thing for me in this course is trying to understand the content as thoroughly as possible.Q1.4When I have the opportunity in this class, I choose course assignments that I can learn from even if they do not guarantee a good grade.Extrinsic goal orientationQ2.1Getting a good grade in this class is the most satisfying thing for me right now.Q2.2The most important thing for me right now is improving my overall grade point average, so my main concern in this class is getting a good grade.Q2.3If I can, I want to get better grades in this class than most of the other students.Q2.4I want to do well in this class because it is important to show my ability to my family, friends, employer, or others.Control of learning beliefsQ3.1If I study in appropriate ways, then I will be able to learn the material in this course.Q3.2It is my own fault if I do not learn the material in this course.Q3.3If I try hard enough, then I will understand the course material.Q3.4If I do not understand the course material, it is because I did not try hard enough.Self-efficacy for learning and performanceQ4.1I believe I will receive an excellent grade in this class.Q4.2I’m certain I can understand the most difficult material presented in the readings for this course.Q4.3I’m confident I can learn the basic concepts taught in this course.Q4.4I’m confident I can understand the most complex material presented by the instructor in this course.Q4.5I’m confident I can do an excellent job on the assignments and tests in this course.Q4.6Considering the difficulty of this course, the teacher, and my skills, I think I will do well in this class.Intrinsic goal orientation refers to the degree to which individuals participate in target activities, perform particular behaviors, or achieve specific goals for inner satisfaction [45]. Extrinsic goal orientation refers to the degree to which individuals participate in target activities, perform particular behaviors, or achieve specific goals for external stimuli, e.g., grades, rewards, performance, evaluation by others, or competition [45]. Control of learning belief reflects how individuals believe that their efforts can contribute to promising outcomes [45]. Self-efficacy for learning and performance reflects individuals’ appraisal and judgements of their ability to master and accomplish particular tasks [45].


### 3.3. Experiment Procedures

The overall procedures involved Kahoot!, adapted CET-6, and adapted MSLQ (see Figure2).


#### 3.3.1. Basic Procedures

We conducted this experiment over six weeks (from July 19th, 2021, to August 29th, 2021). Participants divided in three groups attended the course College English five times a week. Each course lasted about 100 minutes. Three teachers with more than five years of teaching English parallelly taught the three groups and were the raters of CET-6 pre- and posttest.Before the course began, the participants took the CET-6 pretest. We collected and recorded participants’ pretest scores. Then, the participants in three groups began to take the course employing different pedagogies. The teacher in the first experimental group utilized gamification through the “Live Host” mode of Kahoot!. The teacher in the second experimental group utilized game-based learning through the “self-paced kahoots” mode of Kahoot!. The teacher in the control group adopted the traditional didactic pedagogy without any game-related elements. After the 6-week course, the participants took the CET-6 posttest to assess their English proficiency as learning achievement and took the postcourse MSLQ to quantitatively assessment their motivation.


#### 3.3.2. Empirical Procedures in the Gamification Group

In the first experimental group, the teacher regarded Kahoot! as a separable game-related element in class. After teaching new knowledge, the teacher started the “Live Host” mode providing live and competitive activity. The “Live Host” gamified the quizzes to consolidate participants’ understanding and mastery of knowledge.The quizzes involve two parts: multiple choice questions and open-ended question. Participants answered the multiple choice questions and gained scores based on answering accuracy. Each time all the participants have finished one multiple choice question, the teacher explained the key concepts and resolved the misunderstanding based on participants’ accuracy and the frequencies of each answer choice.Once the teacher confirmed that most students understood all the key concepts and knowledge from the multiple choice questions, the teacher set one open-ended question about language usage. While submitting the answer to the dashboard, the participants commented on the submitted answers and shared their ideas with others. Meanwhile, the teacher evaluated the submissions and guided the participants on in-depth discussions.After the “Host Live” section, Kahoot! presented a podium for three students that have gained the most scores. Then, the course ended, and the teacher repeated this cycle for all courses in the gamification group.


#### 3.3.3. Empirical Procedures in the Game-Based Learning Group

In the second experimental group, the teacher regarded Kahoot! as a nonseparable game along the courses. Having taught new knowledge, the teacher assigned the “self-paced kahoots” mode to consolidate participants’ understanding and mastery of knowledge. The “self-paced kahoots” mode involved the flashcards section for learning and practice section for practicing.The “self-paced kahoots” mode enabled participants to learn new knowledge. Participants answered the quizzes tailored to their answering speed. Meanwhile, the teacher received participants’ submissions synchronically. Based on participants’ accuracy, the teacher explained the key concepts and resolved the misunderstanding.An open-ended question follow the “self-paced” mode. Having confirmed that most students understood all the key concepts in the “self-paced kahoots” mode, the teacher assigned an open-ended question for participants’ language usage. While the participants were discussing with others, the teacher listened to their group discussion contents and guided them on in-depth discussions.After the discussion, the teacher called some volunteers to present their ideas. Facing the points worthy of further in-depth discussions, the teacher could record them and encourage interactive discussions. Then, the teacher and participants commented on the presentations, and the course ended. The teacher repeated this cycle for all courses in the game-based learning group.


#### 3.3.4. Empirical Procedure in the Control Group

With the teaching plan same as those in the experimental groups, the teacher taught the participants in the control group using slides, during which the teacher asked the participants and listened to their responses. Having taught all the new knowledge, the teacher assigned an open-ended question to participants for consolidation of their language usage. After the discussion, the teacher picked some volunteers to present their answers to the question. The course ended after the teacher summarized the key concepts and resolved the understanding. Then, the teacher repeated this cycle for all courses in the control group.


### 3.4. Supplementary Materials and Statistical Analyses

We employed IBM® SPSS® Statistics 26 to perform statistical analyses. We collected two datasets: “Data for H1” and “Data for H2” to test the two hypotheses. The datasets are the supplementary materials (availablehere) for this study.“Data for H1” contains the pre- and posttest scores from three groups. The variable “Pedagogy” means the pedagogies that participants received during the empirical procedures (1: gamification; 2: game-based learning; and 3: traditional didactic). The variables “Pretest” and “Posttest” are participants’ scores of the CET-6 pre- and posttest. The variable “Gender” demonstrates participants’ genders (1: female and 2: male).We investigated the impacts of game-related pedagogies on learning achievement by comparing posttest scores (“Posttest”) for three “Pedagogy” groups. However, participants’ English foundation might have interfering influences on the results. Thus, we conducted ANCOVA (analysis of covariance) by setting “Pretest” as the covariate, “Posttest” as the dependent variable, and “Pedagogy” as the independent variable to exclude the impacts of pretest scores on the experiment results ([46], pp. 204).“Data for H2” contains participants’ scores on the question items about motivation. The variable “Pedagogy” means the pedagogies that participants received in the empirical procedures (1: gamification; 2: game-based learning, and 3: traditional didactic). The variables from “Q1.1” to “Q4.6” demonstrate participants’ scores of the 5-Likert MSLQ questions on motivation-related subscales: intrinsic goal orientation (“Q1.1”-“Q1.4”), extrinsic goal orientation (“Q2.1”-“Q2.4”), control of learning beliefs (“Q3.1”-“3.4”), and self-efficacy for learning and performance (“Q4.1”-“Q4.6”).We investigated the impacts of game-related pedagogies on motivation by comparing participants’ scores in question items (“Q1.1”-“Q4.6”) for three “Pedagogy” groups. Based on the reliability and validity of the questionnaire, we performed ANOVA (analysis of variance) to identify whether gamification and game-based learning exert different effects on motivation based on motivation-related subscales ([46], pp. 161).


## 3.1. Research Participants

This study involved 120 participants (86 female students and 34 male students) who were seniors majoring in fine arts from universities. The participants attended the courses of English for general purposes to enhance their English proficiency.We randomly divided these participants into three groups similar in scales and distributions of female and male participants (see Figure2). The first experimental group involved 40 participants, including 29 female students and 11 male students. The second experimental group involved 40 participants, including 30 female students and ten male students. The control group involved 40 participants, including 27 female students and 13 male students.Figure 2
The experiment procedures assessing the impacts of game-related pedagogies on learning achievement and motivation.Participants in the first experimental group attended the English courses employing gamification. Participants in the second experimental group attended the courses employing game-based learning. Participants in the control group attended the courses adopting traditional didactic pedagogy without any game-related designs. Setting a control group could demonstrate whether game-related pedagogies have significant positive impacts on learning achievement and motivation. Setting two experimental groups enabled us to compare the quantitative assessments of learning achievement and motivation for gamification and game-based learning.


## 3.2. Research Instruments

Kahoot! was the platform to perform gamification and game-based learning. We employed the English proficiency test and motivation-related questionnaire as the instruments for quantitative assessments of learning achievement and motivation, respectively.


### 3.2.1. Kahoot! as the Platform for Gamification and Game-Based Learning

We utilized Kahoot! (https://kahoot.com) to perform game-related pedagogies. Initially developed in 2012 and published in 2013, Kahoot! is a game-based student response system (GSRS) transforming the classroom into a game show [42–44].Past studies revealed the positive effects of Kahoot!. Most learners stated that Kahoot! could enhance their motivation, enjoyment, excitement, engagement, learning experiences, and learning efficiency [44]. Besides, Kahoot! provided a user-friendly atmosphere, attractive images, and dynamic music that could stimulate learners’ interest in learning [43]. Kahoot! could also positively influence academic performance and stimulate active attitudes to learning [25, 26].We employed Kahoot! in two ways to perform gamification and game-based learning. Since gamification requires separable game elements rather than a complete game, the first experimental group has used the “Host Live” mode involving the elements of game shows: points, leaderboards, count-down tick, and podium for immersive playful activities. Since game-based learning requires complete game(s) that play an essential auxiliary role in pedagogical practices, the second experimental group has employed the “self-paced kahoots” mode as an independent and complete game design involving cycles of learning, reciting, and practicing.


### 3.2.2. College English-6 (CET-6) for Quantitatively Assessing Learning Achievement

We employed adapted versions of College English Test-6 (CET-6) to quantitatively assess participants’ learning achievement in English proficiency. The authoritative roles of College English Test-4 (CET-4) and College English-6 (CET-6) in China contribute to the relatively reliable and persuasive results about English proficiency from CET-6.We employed adapted CET-6 in the forms of pretest and posttest to assess participants’ English foundation before attending the courses and English proficiency after attending the courses, respectively. The total score was 100 for both pretest and posttest. Both pretest and posttest contained ten cloze questions, ten paragraph matching questions, ten multiple-choice questions, one Chinese to English translation task, and one writing task from CET-6.


### 3.2.3. Motivated Strategies for Learning Questionnaire (MSLQ) for Quantitatively Assessing Motivation

We implemented the adapted Motivated Strategies for Learning Questionnaire (MSLQ) in the study of Pintrich, Smith, García, and McKeachie [45] (see Table 1). The adapted questionnaire consisted of 18 questions scored with a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The 18 questions belonged to four subscales: extrinsic goal orientation (Q1.1-Q1.4), intrinsic goal orientation (Q2.1-Q2.4), control of learning beliefs (Q3.1-3.4), and self-efficacy for learning and performance (Q4.1-Q4.6).Table 1
The adapted version of Motivated Strategies for Learning Questionnaire (MSLQ) in the study of Pintrich, Smith, García, and McKeachie [45].
Motivation-related subscalesNo.Questions scored with a 5-point Likert scaleIntrinsic goal orientationQ1.1In a class like this, I prefer course material that really changes me so I can learn new things.Q1.2In a class like this, I prefer course material that arouses my curiosity, even if it is difficult to learn.Q1.3The most satisfying thing for me in this course is trying to understand the content as thoroughly as possible.Q1.4When I have the opportunity in this class, I choose course assignments that I can learn from even if they do not guarantee a good grade.Extrinsic goal orientationQ2.1Getting a good grade in this class is the most satisfying thing for me right now.Q2.2The most important thing for me right now is improving my overall grade point average, so my main concern in this class is getting a good grade.Q2.3If I can, I want to get better grades in this class than most of the other students.Q2.4I want to do well in this class because it is important to show my ability to my family, friends, employer, or others.Control of learning beliefsQ3.1If I study in appropriate ways, then I will be able to learn the material in this course.Q3.2It is my own fault if I do not learn the material in this course.Q3.3If I try hard enough, then I will understand the course material.Q3.4If I do not understand the course material, it is because I did not try hard enough.Self-efficacy for learning and performanceQ4.1I believe I will receive an excellent grade in this class.Q4.2I’m certain I can understand the most difficult material presented in the readings for this course.Q4.3I’m confident I can learn the basic concepts taught in this course.Q4.4I’m confident I can understand the most complex material presented by the instructor in this course.Q4.5I’m confident I can do an excellent job on the assignments and tests in this course.Q4.6Considering the difficulty of this course, the teacher, and my skills, I think I will do well in this class.Intrinsic goal orientation refers to the degree to which individuals participate in target activities, perform particular behaviors, or achieve specific goals for inner satisfaction [45]. Extrinsic goal orientation refers to the degree to which individuals participate in target activities, perform particular behaviors, or achieve specific goals for external stimuli, e.g., grades, rewards, performance, evaluation by others, or competition [45]. Control of learning belief reflects how individuals believe that their efforts can contribute to promising outcomes [45]. Self-efficacy for learning and performance reflects individuals’ appraisal and judgements of their ability to master and accomplish particular tasks [45].


## 3.2.1. Kahoot! as the Platform for Gamification and Game-Based Learning

We utilized Kahoot! (https://kahoot.com) to perform game-related pedagogies. Initially developed in 2012 and published in 2013, Kahoot! is a game-based student response system (GSRS) transforming the classroom into a game show [42–44].Past studies revealed the positive effects of Kahoot!. Most learners stated that Kahoot! could enhance their motivation, enjoyment, excitement, engagement, learning experiences, and learning efficiency [44]. Besides, Kahoot! provided a user-friendly atmosphere, attractive images, and dynamic music that could stimulate learners’ interest in learning [43]. Kahoot! could also positively influence academic performance and stimulate active attitudes to learning [25, 26].We employed Kahoot! in two ways to perform gamification and game-based learning. Since gamification requires separable game elements rather than a complete game, the first experimental group has used the “Host Live” mode involving the elements of game shows: points, leaderboards, count-down tick, and podium for immersive playful activities. Since game-based learning requires complete game(s) that play an essential auxiliary role in pedagogical practices, the second experimental group has employed the “self-paced kahoots” mode as an independent and complete game design involving cycles of learning, reciting, and practicing.


## 3.2.2. College English-6 (CET-6) for Quantitatively Assessing Learning Achievement

We employed adapted versions of College English Test-6 (CET-6) to quantitatively assess participants’ learning achievement in English proficiency. The authoritative roles of College English Test-4 (CET-4) and College English-6 (CET-6) in China contribute to the relatively reliable and persuasive results about English proficiency from CET-6.We employed adapted CET-6 in the forms of pretest and posttest to assess participants’ English foundation before attending the courses and English proficiency after attending the courses, respectively. The total score was 100 for both pretest and posttest. Both pretest and posttest contained ten cloze questions, ten paragraph matching questions, ten multiple-choice questions, one Chinese to English translation task, and one writing task from CET-6.


## 3.2.3. Motivated Strategies for Learning Questionnaire (MSLQ) for Quantitatively Assessing Motivation

We implemented the adapted Motivated Strategies for Learning Questionnaire (MSLQ) in the study of Pintrich, Smith, García, and McKeachie [45] (see Table 1). The adapted questionnaire consisted of 18 questions scored with a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The 18 questions belonged to four subscales: extrinsic goal orientation (Q1.1-Q1.4), intrinsic goal orientation (Q2.1-Q2.4), control of learning beliefs (Q3.1-3.4), and self-efficacy for learning and performance (Q4.1-Q4.6).Table 1
The adapted version of Motivated Strategies for Learning Questionnaire (MSLQ) in the study of Pintrich, Smith, García, and McKeachie [45].
Motivation-related subscalesNo.Questions scored with a 5-point Likert scaleIntrinsic goal orientationQ1.1In a class like this, I prefer course material that really changes me so I can learn new things.Q1.2In a class like this, I prefer course material that arouses my curiosity, even if it is difficult to learn.Q1.3The most satisfying thing for me in this course is trying to understand the content as thoroughly as possible.Q1.4When I have the opportunity in this class, I choose course assignments that I can learn from even if they do not guarantee a good grade.Extrinsic goal orientationQ2.1Getting a good grade in this class is the most satisfying thing for me right now.Q2.2The most important thing for me right now is improving my overall grade point average, so my main concern in this class is getting a good grade.Q2.3If I can, I want to get better grades in this class than most of the other students.Q2.4I want to do well in this class because it is important to show my ability to my family, friends, employer, or others.Control of learning beliefsQ3.1If I study in appropriate ways, then I will be able to learn the material in this course.Q3.2It is my own fault if I do not learn the material in this course.Q3.3If I try hard enough, then I will understand the course material.Q3.4If I do not understand the course material, it is because I did not try hard enough.Self-efficacy for learning and performanceQ4.1I believe I will receive an excellent grade in this class.Q4.2I’m certain I can understand the most difficult material presented in the readings for this course.Q4.3I’m confident I can learn the basic concepts taught in this course.Q4.4I’m confident I can understand the most complex material presented by the instructor in this course.Q4.5I’m confident I can do an excellent job on the assignments and tests in this course.Q4.6Considering the difficulty of this course, the teacher, and my skills, I think I will do well in this class.Intrinsic goal orientation refers to the degree to which individuals participate in target activities, perform particular behaviors, or achieve specific goals for inner satisfaction [45]. Extrinsic goal orientation refers to the degree to which individuals participate in target activities, perform particular behaviors, or achieve specific goals for external stimuli, e.g., grades, rewards, performance, evaluation by others, or competition [45]. Control of learning belief reflects how individuals believe that their efforts can contribute to promising outcomes [45]. Self-efficacy for learning and performance reflects individuals’ appraisal and judgements of their ability to master and accomplish particular tasks [45].


## 3.3. Experiment Procedures

The overall procedures involved Kahoot!, adapted CET-6, and adapted MSLQ (see Figure2).


### 3.3.1. Basic Procedures

We conducted this experiment over six weeks (from July 19th, 2021, to August 29th, 2021). Participants divided in three groups attended the course College English five times a week. Each course lasted about 100 minutes. Three teachers with more than five years of teaching English parallelly taught the three groups and were the raters of CET-6 pre- and posttest.Before the course began, the participants took the CET-6 pretest. We collected and recorded participants’ pretest scores. Then, the participants in three groups began to take the course employing different pedagogies. The teacher in the first experimental group utilized gamification through the “Live Host” mode of Kahoot!. The teacher in the second experimental group utilized game-based learning through the “self-paced kahoots” mode of Kahoot!. The teacher in the control group adopted the traditional didactic pedagogy without any game-related elements. After the 6-week course, the participants took the CET-6 posttest to assess their English proficiency as learning achievement and took the postcourse MSLQ to quantitatively assessment their motivation.


### 3.3.2. Empirical Procedures in the Gamification Group

In the first experimental group, the teacher regarded Kahoot! as a separable game-related element in class. After teaching new knowledge, the teacher started the “Live Host” mode providing live and competitive activity. The “Live Host” gamified the quizzes to consolidate participants’ understanding and mastery of knowledge.The quizzes involve two parts: multiple choice questions and open-ended question. Participants answered the multiple choice questions and gained scores based on answering accuracy. Each time all the participants have finished one multiple choice question, the teacher explained the key concepts and resolved the misunderstanding based on participants’ accuracy and the frequencies of each answer choice.Once the teacher confirmed that most students understood all the key concepts and knowledge from the multiple choice questions, the teacher set one open-ended question about language usage. While submitting the answer to the dashboard, the participants commented on the submitted answers and shared their ideas with others. Meanwhile, the teacher evaluated the submissions and guided the participants on in-depth discussions.After the “Host Live” section, Kahoot! presented a podium for three students that have gained the most scores. Then, the course ended, and the teacher repeated this cycle for all courses in the gamification group.


### 3.3.3. Empirical Procedures in the Game-Based Learning Group

In the second experimental group, the teacher regarded Kahoot! as a nonseparable game along the courses. Having taught new knowledge, the teacher assigned the “self-paced kahoots” mode to consolidate participants’ understanding and mastery of knowledge. The “self-paced kahoots” mode involved the flashcards section for learning and practice section for practicing.The “self-paced kahoots” mode enabled participants to learn new knowledge. Participants answered the quizzes tailored to their answering speed. Meanwhile, the teacher received participants’ submissions synchronically. Based on participants’ accuracy, the teacher explained the key concepts and resolved the misunderstanding.An open-ended question follow the “self-paced” mode. Having confirmed that most students understood all the key concepts in the “self-paced kahoots” mode, the teacher assigned an open-ended question for participants’ language usage. While the participants were discussing with others, the teacher listened to their group discussion contents and guided them on in-depth discussions.After the discussion, the teacher called some volunteers to present their ideas. Facing the points worthy of further in-depth discussions, the teacher could record them and encourage interactive discussions. Then, the teacher and participants commented on the presentations, and the course ended. The teacher repeated this cycle for all courses in the game-based learning group.


### 3.3.4. Empirical Procedure in the Control Group

With the teaching plan same as those in the experimental groups, the teacher taught the participants in the control group using slides, during which the teacher asked the participants and listened to their responses. Having taught all the new knowledge, the teacher assigned an open-ended question to participants for consolidation of their language usage. After the discussion, the teacher picked some volunteers to present their answers to the question. The course ended after the teacher summarized the key concepts and resolved the understanding. Then, the teacher repeated this cycle for all courses in the control group.


## 3.3.1. Basic Procedures

We conducted this experiment over six weeks (from July 19th, 2021, to August 29th, 2021). Participants divided in three groups attended the course College English five times a week. Each course lasted about 100 minutes. Three teachers with more than five years of teaching English parallelly taught the three groups and were the raters of CET-6 pre- and posttest.Before the course began, the participants took the CET-6 pretest. We collected and recorded participants’ pretest scores. Then, the participants in three groups began to take the course employing different pedagogies. The teacher in the first experimental group utilized gamification through the “Live Host” mode of Kahoot!. The teacher in the second experimental group utilized game-based learning through the “self-paced kahoots” mode of Kahoot!. The teacher in the control group adopted the traditional didactic pedagogy without any game-related elements. After the 6-week course, the participants took the CET-6 posttest to assess their English proficiency as learning achievement and took the postcourse MSLQ to quantitatively assessment their motivation.


## 3.3.2. Empirical Procedures in the Gamification Group

In the first experimental group, the teacher regarded Kahoot! as a separable game-related element in class. After teaching new knowledge, the teacher started the “Live Host” mode providing live and competitive activity. The “Live Host” gamified the quizzes to consolidate participants’ understanding and mastery of knowledge.The quizzes involve two parts: multiple choice questions and open-ended question. Participants answered the multiple choice questions and gained scores based on answering accuracy. Each time all the participants have finished one multiple choice question, the teacher explained the key concepts and resolved the misunderstanding based on participants’ accuracy and the frequencies of each answer choice.Once the teacher confirmed that most students understood all the key concepts and knowledge from the multiple choice questions, the teacher set one open-ended question about language usage. While submitting the answer to the dashboard, the participants commented on the submitted answers and shared their ideas with others. Meanwhile, the teacher evaluated the submissions and guided the participants on in-depth discussions.After the “Host Live” section, Kahoot! presented a podium for three students that have gained the most scores. Then, the course ended, and the teacher repeated this cycle for all courses in the gamification group.


## 3.3.3. Empirical Procedures in the Game-Based Learning Group

In the second experimental group, the teacher regarded Kahoot! as a nonseparable game along the courses. Having taught new knowledge, the teacher assigned the “self-paced kahoots” mode to consolidate participants’ understanding and mastery of knowledge. The “self-paced kahoots” mode involved the flashcards section for learning and practice section for practicing.The “self-paced kahoots” mode enabled participants to learn new knowledge. Participants answered the quizzes tailored to their answering speed. Meanwhile, the teacher received participants’ submissions synchronically. Based on participants’ accuracy, the teacher explained the key concepts and resolved the misunderstanding.An open-ended question follow the “self-paced” mode. Having confirmed that most students understood all the key concepts in the “self-paced kahoots” mode, the teacher assigned an open-ended question for participants’ language usage. While the participants were discussing with others, the teacher listened to their group discussion contents and guided them on in-depth discussions.After the discussion, the teacher called some volunteers to present their ideas. Facing the points worthy of further in-depth discussions, the teacher could record them and encourage interactive discussions. Then, the teacher and participants commented on the presentations, and the course ended. The teacher repeated this cycle for all courses in the game-based learning group.


## 3.3.4. Empirical Procedure in the Control Group

With the teaching plan same as those in the experimental groups, the teacher taught the participants in the control group using slides, during which the teacher asked the participants and listened to their responses. Having taught all the new knowledge, the teacher assigned an open-ended question to participants for consolidation of their language usage. After the discussion, the teacher picked some volunteers to present their answers to the question. The course ended after the teacher summarized the key concepts and resolved the understanding. Then, the teacher repeated this cycle for all courses in the control group.


## 3.4. Supplementary Materials and Statistical Analyses

We employed IBM® SPSS® Statistics 26 to perform statistical analyses. We collected two datasets: “Data for H1” and “Data for H2” to test the two hypotheses. The datasets are the supplementary materials (availablehere) for this study.“Data for H1” contains the pre- and posttest scores from three groups. The variable “Pedagogy” means the pedagogies that participants received during the empirical procedures (1: gamification; 2: game-based learning; and 3: traditional didactic). The variables “Pretest” and “Posttest” are participants’ scores of the CET-6 pre- and posttest. The variable “Gender” demonstrates participants’ genders (1: female and 2: male).We investigated the impacts of game-related pedagogies on learning achievement by comparing posttest scores (“Posttest”) for three “Pedagogy” groups. However, participants’ English foundation might have interfering influences on the results. Thus, we conducted ANCOVA (analysis of covariance) by setting “Pretest” as the covariate, “Posttest” as the dependent variable, and “Pedagogy” as the independent variable to exclude the impacts of pretest scores on the experiment results ([46], pp. 204).“Data for H2” contains participants’ scores on the question items about motivation. The variable “Pedagogy” means the pedagogies that participants received in the empirical procedures (1: gamification; 2: game-based learning, and 3: traditional didactic). The variables from “Q1.1” to “Q4.6” demonstrate participants’ scores of the 5-Likert MSLQ questions on motivation-related subscales: intrinsic goal orientation (“Q1.1”-“Q1.4”), extrinsic goal orientation (“Q2.1”-“Q2.4”), control of learning beliefs (“Q3.1”-“3.4”), and self-efficacy for learning and performance (“Q4.1”-“Q4.6”).We investigated the impacts of game-related pedagogies on motivation by comparing participants’ scores in question items (“Q1.1”-“Q4.6”) for three “Pedagogy” groups. Based on the reliability and validity of the questionnaire, we performed ANOVA (analysis of variance) to identify whether gamification and game-based learning exert different effects on motivation based on motivation-related subscales ([46], pp. 161).


## 4. Results

This section reports the statistical findings based on which we test the hypotheses.


### 4.1. Test the Hypothesis that Gamification and Game-Based Learning Exert Differently Positive Effects on Learning Achievement

Before we began ANCOVA based on “Data for H1”, we checked whether the data satisfied the four requisites of ANCOVA: normal distribution, homogeneity of regression slopes, insignificant interaction between the independent variable and the covariate, and the equality of the variance in the dependent variable group.


#### 4.1.1. Four Requisites for ANCOVA that “Data for H1” Satisfied

The first requisite is that the variables should be normally distributed. We performed the one-sample Kolmogorov-Smirnov test to investigate whether the variable groups “Pretest” and “Posttest” were normally distributed. The significancelevels>0.05 of .200 for both “Pretest” and “Posttest” variable groups suggested that the distributions of values were approximately consistent with the normal curves, confirming the normal distributions ([46], pp. 125). Thus, the variable groups “Pretest” and “Posttest” fulfilled the requisite of normal distribution.The second requisite is that regression slopes should be homogeneous so that the relationships between the dependent variable (“Posttest”) and the covariate (“Pretest”) are similar in all the treatment groups (the three “Pedagogy” groups) ([46], pp. 208). We used the grouped scatter chart to investigate whether the linear relationships between “Posttest” and “Pretest” present the homogeneous regression slopes in the three pedagogical approach conditions ([46], pp. 223). The grouped scatter chart (see Figure 3) presented that the slopes were 0.40, 0.39, and 0.38 for the lines denoting gamification, game-based learning, and traditional pedagogy, respectively. Since the three lines were relatively parallel, the linear relations between dependent variable and covariate were homogeneous in all the treat groups, fulfilling the requisite of homogeneity of regression slopes ([46], pp. 208).Figure 3
The grouped scatter chart presenting the homogeneous regression slopes of the linear relations between pretest and posttest scores by three pedagogies.The third requisite is that the interaction between the independent variable and the covariate (“Pedagogy∗Pretest”) should be insignificant to exclude the interfering effects of the covariate ([46], pp. 208). The test of between-subjects effects revealed that the significance levels of “Pedagogy,” “Pretest scores,” and “Pedagogy∗Pretest” are 0.707, 0.000, and 0.972, respectively. If not set as the covariate, the “Pretest” variable group would exert interfering effects (sig.=0.000<0.05). Notably, the insignificant interaction between “Pedagogy” and “Pretest” (sig.=0.972>0.05) could exclude the interfering effects of “Pretest scores” ([46], pp. 210). Thus, “Data for H1” satisfied the requisite of the insignificant interaction between the independent variable and the covariate.The fourth requisite is the variance equality for the dependent values (“Posttest”) in the treatment groups (“Pedagogy”) ([46], pp. 213). Levene’s test revealed the significance level>0.05 of 0.107, confirming that the “Posttest” variable group did not violate the assumption of the variance equality for all the treatment groups ([46], pp. 213). Thus, the variances of dependent values were relatively equal for all the three treatment groups.Overall, “Data for H1” satisfied all the four requisites for ANCOVA. We conducted ANCOVA to investigate compare the effects on learning achievement for gamification and game-based learning.


#### 4.1.2. ANCOVA Comparing the Effects on Learning Achievement for Gamification and Game-Based Learning

ANCOVA reported the average dependent values in all the treatment groups. The descriptive statistics (see Table2) revealed that two experimental groups (N=40, M=79.301, and SD=1.258 for the gamification group; N=40, M=77.473, and SD=1.262 for the game-based learning group) had higher average CET-6 posttest scores than the control group (N=40, M=74.026, and SD=1.258). Specifically speaking, the gamification group (M=79.301) had higher average CET-6 posttest scores than the game-based learning group (M=77.473).Table 2
ANCOVA estimates comparing the average CET-6 posttest scores for the three treatment groups.
Dependent variable: posttest scoresPedagogyMeanStd. error95% confidence intervalLower boundUpper boundGamification79.301a1.25876.81181.792Game-based learning77.473a1.26274.97479.971Traditional didactic74.026a1.25871.53476.518aCovariates appearing in the model are evaluated at the following values: pretestscores=52.5917.ANCOVA also reported whether the average CET-6 posttest scores significantly differed among the three treatment groups based on the covariate of the CET-6 pretest scores. The between-subjects effects test (see Table3) revealed that the average CET-6 posttest scores were significantly different among the three treatment groups (sig.=0.013<0.05). That finding confirmed that game-related pedagogies could significantly influence learning achievement reflected by the average CET-6 posttests.Table 3
The between-subjects effects test examining the significance levels.
Dependent variable: posttest scoresSourceType III sum of squaresdfMean squareFSig.Corrected model8458.939a32819.64644.6430.000Intercept51879.073151879.073821.3950.000Pedagogy573.5722286.7864.5410.013Pretest7906.47217906.472125.1820.000Error7326.52811663.160Total726034.000120Corrected total15785.467119aRsquared=0.536 (adjustedRsquared=0.524).The pairwise comparisons (see Table4) reported the differences between treatment groups in detail. The significant difference in average CET-6 posttest scores between the gamification and control groups (sig.=0.004<0.05) confirmed the significant positive effects of gamification on learning achievement. However, the relatively insignificant difference between the game-based and control groups (sig.=0.056>0.05) suggested that game-based learning exerted insignificantly positive effects on learning achievement. Thus, even if both gamification and game-based learning positively influenced learning achievement reflected by average CET-6 posttest scores, gamification exerted relatively more significant effects on learning achievement than game-based learning.Table 4
Pairwise comparisons examining the significance level for the differences in CET-6 posttest scores for the three treatment groups.
Dependent variablePosttest scores(I) Pedagogy(J) PedagogyMean difference (I-J)Std. errorSig.b95% confidence interval for differencebLower boundUpper boundGamificationGame-based learning1.8291.7850.308-1.7065.363Traditional didactic5.275∗1.7770.0041.7558.795Game-based learningGamification-1.8291.7850.308-5.3631.706Traditional didactic3.4461.7860.056-0.0916.984Traditional didacticGamification-5.275∗1.7770.004-8.795-1.755Game-based learning-3.4461.7860.056-6.9840.091Based on estimated marginal means.∗The mean difference is significant at the 0.05 level. bAdjustment for multiple comparisons: least significant difference (equivalent to no adjustments).Based on these findings, we confirmed H1 that gamification and game-based learning exert differently positive effects on learning achievement. Gamification exerted significantly positive effects on learning achievement, while game-based learning exerted insignificantly positive effects on learning achievement.


### 4.2. Test the Hypothesis that Gamification and Game-Based Learning Exert Differently Positive Effects on Motivation

“Data for H2” involved 108 records from the validly answered questionnaires (37 for the gamification group, 35 for the game-based learning group, and 36 for the control group). We first investigated whether “Data for H2” satisfied reliability, validity, and normal distribution for further analyses.


#### 4.2.1. Reliability, Validity, and Normal Distribution that “Data for H2” Satisfied for Further Analyses

Reliability stands for the stability of the result outputs over time ([47], pp. 243). Cronbach’s α is the value to assess the reliability of the data ([47], pp. 251). The overall Cronbach α value is 0.810>0.70 for all the eighteen question items, reflecting overall reliable results ([47], pp. 265). Thus, the data satisfied the demand of reliability.Validity stands for the degree to which the measurements are consistent with the target facets ([48], pp. 14). Factor analysis could reduce the data dimensions by combining variables that approximately assess the same facets ([46], pp. 286). Thus, we performed the factor analysis to examine whether the data could provide valid measurement of the motivation-related subscales.Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) and Bartlett’s Test of Sphericity could identify whether the data are suitable for the factor analysis ([46], pp. 292). The value>0.60 of 0.739 from KMO, the significance level<0.05 of 0.000 from Bartlett’s Test of Sphericity, and the chi-square of 691.547 at df=153 from Bartlett’s Test of Sphericity suggested that we could perform the factor analysis for “Data for H2” ([46], pp. 292).The factor analysis reported the scree plot chart (see Figure4) and the rotated component matrix values (see Table 5). According to the scree plot (see Figure 4), the eigenvalue line steadily decreases from the fifth component and reported four high-loading components standing for the main facets. The rotated component matrix table (see Table 5) demonstrated the correlations between the question items and the four high-loading components ([46], pp. 294). Q1.1-Q1.4, Q2.1-Q2.4, Q3.1-Q3.4, and Q4.1-Q4.6 were most positively correlated with the third, second, fourth, and first components, respectively (see Table 5). These reports confirmed that “Data for H2” involved a valid measurement of motivation-related subscales, satisfying the demand of validity. The first, second, third, and fourth facets were self-efficacy for learning and performance (“Efficacy”), extrinsic goal orientation (“Extrinsic”), intrinsic goal orientation (“Intrinsic”), and control of learning beliefs (“Beliefs”), respectively. Accordingly, we calculated every participant’s average scores for each motivation-related subscale.Figure 4
The scree plot presenting high-loading components for the factor analysis.Table 5
Rotated component matrixa presenting the correlations between question items and high-loading components.
Component1234Q1.10.0510.0620.7790.112Q1.20.0260.0750.716-0.090Q1.30.1250.2310.7770.011Q1.40.1710.0830.7910.002Q2.10.0950.8120.1550.124Q2.20.1030.8640.130-0.034Q2.30.0250.8640.0780.076Q2.40.1540.7890.0990.132Q3.1-0.0620.1440.1240.564Q3.20.0940.126-0.0110.773Q3.30.259-0.032-0.0280.742Q3.40.1100.008-0.0850.767Q4.10.6970.0240.1050.264Q4.20.6610.0430.0660.233Q4.30.7410.1500.0700.061Q4.40.706-0.034-0.032-0.027Q4.50.5920.1820.0500.034Q4.60.7040.0570.194-0.036Extraction method: principal component analysis. Rotation method: Varimax with Kaiser normalization.aRotation converged in 5 iterations.As for normal distribution, we performed the Kolmogorov-Smirnov (K-S) test. We conducted the K-S test without the Lilliefors correction to reduce sensitivity to departures from normality ([46], pp. 254). The test without the Lilliefors correction suggested that “Efficacy,” “Extrinsic,” “Intrinsic,” and “Beliefs” have reached the significant levels>0.05 of 0.123, 0.073, 0.106, and 0.115, respectively. Thus, motivation-related subscale variables satisfied the demand of normal distribution.Thus far, “Data for H2” satisfied the conditions of reliability, validity, and normal distribution. We could conduct ANOVA to compare the effects on motivation for gamification and game-based learning.


#### 4.2.2. ANOVA Comparing the Effects on Motivation for Gamification and Game-Based Learning

We first checked the reports about the average scores in the four motivation-related subscales for the three treatment groups (see Table6). Generally speaking, compared with the control group, groups employing game-related pedagogies presented relatively higher average scores in motivation-related subscales. The average score in “Intrinsic” was slightly lower for the game-based learning group than the control group. ANOVA would report whether the average scores in motivation-related subscales were significantly different among the three treatment groups.Table 6
Reports about the average scores in motivation-related subscales for the three treatment groups.
PedagogyEfficacyExtrinsicIntrinsicBeliefsGamificationMean3.69823.75003.85143.4527N37373737Std. deviation0.624650.803640.769330.63694Game-based learningMean3.60002.91433.60713.1643N35353535Std. deviation0.506820.947020.610220.63577Traditional didacticMean3.39812.67363.75692.7361N36363636Std. deviation0.611110.936980.747580.85762TotalMean3.56643.12043.74073.1204N108108108108Std. deviation0.592411.003780.714440.77073The ANOVA results (see Table7) revealed that the average scores in “Extrinsic” and “Beliefs” were significantly different among the three treatment groups due to the significance value<0.05 of 0.000 ([46], pp. 167). However, the average scores in “Efficacy” and “Intrinsic” were insignificantly different among the treatment groups due to the levels>0.05 of 0.088 and 0.348, respectively ([46], pp. 167). Thus, game-related pedagogies would significantly influence participants’ scores in extrinsic goal orientation and control of learning beliefs, whereas game-related pedagogies would insignificantly influence the scores in self-efficacy for learning and performance and intrinsic goal orientation. The relatively lower average score in “Intrinsic” for the game-based learning group than the control group would not mean the counterproductive effects of game-based learning due to the insignificant differences in “Intrinsic” average scores for the three groups.Table 7
ANOVA investigating the effects of game-related approaches on motivation in terms of the four motivation-related subscales.
Sum of squaresdfMean squareFSig.EfficacyBetween groups1.70120.8512.4910.088Within groups35.8511050.341Total37.552107ExtrinsicBetween groups23.340211.67014.5060.000Within groups84.4701050.804Total107.810107IntrinsicBetween groups1.08720.5431.0660.348Within groups53.5291050.510Total54.616107BeliefsBetween groups9.47024.7359.1910.000Within groups54.0911050.515Total63.560107The multiple comparisons (see Table8) reported the differences among the treatment groups in detail and identified which pair(s) of means contributed to the significant value(s) ([46], pp. 163). The difference in average scores in “Efficacy” was significant between the gamification group and the control group (sig.=0.000<0.05), while the difference was insignificant between the game-based learning group and the control group (sig.=0.530>0.05). These findings suggested that gamification would significantly enhance extrinsic goal orientation and that game-based learning would insignificantly enhance extrinsic goal orientation. Besides, the difference in the average scores in “Beliefs” was significant between the gamification group and the control group (sig.=0.000<0.05), and the difference was also significant between the game-based learning group and the control group (sig.=0.047<0.05). These findings suggested that gamification would exert more significant effects on control of learning beliefs despite the overall positive effects on control of learning beliefs for both gamification and game-based learning.Table 8
Multiples comparisons the significance level for the differences in participants’ scoring motivation-related subscales for the three treatment groups.
ScheffeDependent variable(I) Pedagogy(J) PedagogyMean difference (I-J)Std. errorSig.95% confidence intervalLower boundUpper boundEfficacyGamificationGame-based learning0.098200.137780.776-0.24390.4403Traditional didactic0.300050.136790.095-0.03960.6397Game-based learningGamification-0.098200.137780.776-0.44030.2439Traditional didactic0.201850.138710.351-0.14260.5463Traditional didacticGamification-0.300050.136790.095-0.63970.0396Game-based learning-0.201850.138710.351-0.54630.1426ExtrinsicGamificationGame-based learning0.83571∗0.211490.0010.31061.3609Traditional didactic1.07639∗0.209970.0000.55501.5978Game-based learningGamification-0.83571∗0.211490.001-1.3609-0.3106Traditional didactic0.240670.212910.530-0.28800.7694Traditional didacticGamification-1.07639∗0.209970.000-1.5978-0.5550Game-based learning-0.240670.212910.530-0.76940.2880IntrinsicGamificationGame-based learning0.244210.168360.353-0.17380.6623Traditional didactic0.094410.167150.853-0.32060.5095Game-based learningGamification-0.244210.168360.353-0.66230.1738Traditional didactic-0.149800.169490.678-0.57070.2711Traditional didacticGamification-0.094410.167150.853-0.50950.3206Game-based learning0.149800.169490.678-0.27110.5707BeliefsGamificationGame-based learning0.288420.169240.239-0.13180.7086Traditional didactic0.71659∗0.168030.0000.29941.1338Game-based learningGamification-0.288420.169240.239-0.70860.1318Traditional didactic0.42817∗0.170380.0470.00510.8512Traditional didacticGamification-0.71659∗0.168030.000-1.1338-0.2994Game-based learning-0.42817∗0.170380.047-0.8512-0.0051∗The mean difference is significant at the 0.05 level.The test of variance homogeneity (see Table9) reported whether the ANOVA results were reliable and persuasive. The significance levels>0.05 of 0.597, 0.189, 0.518, and 0.063 suggested the homogeneous variances for “Efficacy,” “Extrinsic,” “Intrinsic,” and “Beliefs,” respectively. Thus, the involved variable groups maintained the assumption of the variance equality, confirming the reliability and persuasiveness of the ANOVA results ([46], pp. 213).Table 9
The variance homogeneity test examining the persuasiveness of the ANOVA results.
Levene statisticdf1df2Sig.EfficacyBased on mean0.51921050.597Based on median0.46221050.631Based on median and with adjusteddf0.462299.7520.631Based on trimmed mean.5202105.596ExtrinsicBased on mean1.6942105.189Based on median2.1212105.125Based on median and with adjusteddf2.121298.608.125Based on trimmed mean1.7772105.174IntrinsicBased on mean.6632105.518Based on median.5732105.565Based on median and with adjusteddf.573299.659.566Based on trimmed mean.6482105.525BeliefsBased on mean2.8432105.063Based on median2.9042105.059Based on median and with adjusteddf2.9042102.658.059Based on trimmed mean2.9042105.059Based on these findings, we confirmed H2 that gamification and game-based learning exert differently positive effects on motivation. Compared with game-based learning, gamification exerts more significantly positive effects on extrinsic goal orientation and control of learning beliefs. Gamification and game-based learning exerted insignificant effects on self-efficacy for learning and performance and intrinsic goal orientation. Gamification and game-based learning exerted different effects on motivation reflected by the four motivation-related subscales.


## 4.1. Test the Hypothesis that Gamification and Game-Based Learning Exert Differently Positive Effects on Learning Achievement

Before we began ANCOVA based on “Data for H1”, we checked whether the data satisfied the four requisites of ANCOVA: normal distribution, homogeneity of regression slopes, insignificant interaction between the independent variable and the covariate, and the equality of the variance in the dependent variable group.


### 4.1.1. Four Requisites for ANCOVA that “Data for H1” Satisfied

The first requisite is that the variables should be normally distributed. We performed the one-sample Kolmogorov-Smirnov test to investigate whether the variable groups “Pretest” and “Posttest” were normally distributed. The significancelevels>0.05 of .200 for both “Pretest” and “Posttest” variable groups suggested that the distributions of values were approximately consistent with the normal curves, confirming the normal distributions ([46], pp. 125). Thus, the variable groups “Pretest” and “Posttest” fulfilled the requisite of normal distribution.The second requisite is that regression slopes should be homogeneous so that the relationships between the dependent variable (“Posttest”) and the covariate (“Pretest”) are similar in all the treatment groups (the three “Pedagogy” groups) ([46], pp. 208). We used the grouped scatter chart to investigate whether the linear relationships between “Posttest” and “Pretest” present the homogeneous regression slopes in the three pedagogical approach conditions ([46], pp. 223). The grouped scatter chart (see Figure 3) presented that the slopes were 0.40, 0.39, and 0.38 for the lines denoting gamification, game-based learning, and traditional pedagogy, respectively. Since the three lines were relatively parallel, the linear relations between dependent variable and covariate were homogeneous in all the treat groups, fulfilling the requisite of homogeneity of regression slopes ([46], pp. 208).Figure 3
The grouped scatter chart presenting the homogeneous regression slopes of the linear relations between pretest and posttest scores by three pedagogies.The third requisite is that the interaction between the independent variable and the covariate (“Pedagogy∗Pretest”) should be insignificant to exclude the interfering effects of the covariate ([46], pp. 208). The test of between-subjects effects revealed that the significance levels of “Pedagogy,” “Pretest scores,” and “Pedagogy∗Pretest” are 0.707, 0.000, and 0.972, respectively. If not set as the covariate, the “Pretest” variable group would exert interfering effects (sig.=0.000<0.05). Notably, the insignificant interaction between “Pedagogy” and “Pretest” (sig.=0.972>0.05) could exclude the interfering effects of “Pretest scores” ([46], pp. 210). Thus, “Data for H1” satisfied the requisite of the insignificant interaction between the independent variable and the covariate.The fourth requisite is the variance equality for the dependent values (“Posttest”) in the treatment groups (“Pedagogy”) ([46], pp. 213). Levene’s test revealed the significance level>0.05 of 0.107, confirming that the “Posttest” variable group did not violate the assumption of the variance equality for all the treatment groups ([46], pp. 213). Thus, the variances of dependent values were relatively equal for all the three treatment groups.Overall, “Data for H1” satisfied all the four requisites for ANCOVA. We conducted ANCOVA to investigate compare the effects on learning achievement for gamification and game-based learning.


### 4.1.2. ANCOVA Comparing the Effects on Learning Achievement for Gamification and Game-Based Learning

ANCOVA reported the average dependent values in all the treatment groups. The descriptive statistics (see Table2) revealed that two experimental groups (N=40, M=79.301, and SD=1.258 for the gamification group; N=40, M=77.473, and SD=1.262 for the game-based learning group) had higher average CET-6 posttest scores than the control group (N=40, M=74.026, and SD=1.258). Specifically speaking, the gamification group (M=79.301) had higher average CET-6 posttest scores than the game-based learning group (M=77.473).Table 2
ANCOVA estimates comparing the average CET-6 posttest scores for the three treatment groups.
Dependent variable: posttest scoresPedagogyMeanStd. error95% confidence intervalLower boundUpper boundGamification79.301a1.25876.81181.792Game-based learning77.473a1.26274.97479.971Traditional didactic74.026a1.25871.53476.518aCovariates appearing in the model are evaluated at the following values: pretestscores=52.5917.ANCOVA also reported whether the average CET-6 posttest scores significantly differed among the three treatment groups based on the covariate of the CET-6 pretest scores. The between-subjects effects test (see Table3) revealed that the average CET-6 posttest scores were significantly different among the three treatment groups (sig.=0.013<0.05). That finding confirmed that game-related pedagogies could significantly influence learning achievement reflected by the average CET-6 posttests.Table 3
The between-subjects effects test examining the significance levels.
Dependent variable: posttest scoresSourceType III sum of squaresdfMean squareFSig.Corrected model8458.939a32819.64644.6430.000Intercept51879.073151879.073821.3950.000Pedagogy573.5722286.7864.5410.013Pretest7906.47217906.472125.1820.000Error7326.52811663.160Total726034.000120Corrected total15785.467119aRsquared=0.536 (adjustedRsquared=0.524).The pairwise comparisons (see Table4) reported the differences between treatment groups in detail. The significant difference in average CET-6 posttest scores between the gamification and control groups (sig.=0.004<0.05) confirmed the significant positive effects of gamification on learning achievement. However, the relatively insignificant difference between the game-based and control groups (sig.=0.056>0.05) suggested that game-based learning exerted insignificantly positive effects on learning achievement. Thus, even if both gamification and game-based learning positively influenced learning achievement reflected by average CET-6 posttest scores, gamification exerted relatively more significant effects on learning achievement than game-based learning.Table 4
Pairwise comparisons examining the significance level for the differences in CET-6 posttest scores for the three treatment groups.
Dependent variablePosttest scores(I) Pedagogy(J) PedagogyMean difference (I-J)Std. errorSig.b95% confidence interval for differencebLower boundUpper boundGamificationGame-based learning1.8291.7850.308-1.7065.363Traditional didactic5.275∗1.7770.0041.7558.795Game-based learningGamification-1.8291.7850.308-5.3631.706Traditional didactic3.4461.7860.056-0.0916.984Traditional didacticGamification-5.275∗1.7770.004-8.795-1.755Game-based learning-3.4461.7860.056-6.9840.091Based on estimated marginal means.∗The mean difference is significant at the 0.05 level. bAdjustment for multiple comparisons: least significant difference (equivalent to no adjustments).Based on these findings, we confirmed H1 that gamification and game-based learning exert differently positive effects on learning achievement. Gamification exerted significantly positive effects on learning achievement, while game-based learning exerted insignificantly positive effects on learning achievement.


## 4.1.1. Four Requisites for ANCOVA that “Data for H1” Satisfied

The first requisite is that the variables should be normally distributed. We performed the one-sample Kolmogorov-Smirnov test to investigate whether the variable groups “Pretest” and “Posttest” were normally distributed. The significancelevels>0.05 of .200 for both “Pretest” and “Posttest” variable groups suggested that the distributions of values were approximately consistent with the normal curves, confirming the normal distributions ([46], pp. 125). Thus, the variable groups “Pretest” and “Posttest” fulfilled the requisite of normal distribution.The second requisite is that regression slopes should be homogeneous so that the relationships between the dependent variable (“Posttest”) and the covariate (“Pretest”) are similar in all the treatment groups (the three “Pedagogy” groups) ([46], pp. 208). We used the grouped scatter chart to investigate whether the linear relationships between “Posttest” and “Pretest” present the homogeneous regression slopes in the three pedagogical approach conditions ([46], pp. 223). The grouped scatter chart (see Figure 3) presented that the slopes were 0.40, 0.39, and 0.38 for the lines denoting gamification, game-based learning, and traditional pedagogy, respectively. Since the three lines were relatively parallel, the linear relations between dependent variable and covariate were homogeneous in all the treat groups, fulfilling the requisite of homogeneity of regression slopes ([46], pp. 208).Figure 3
The grouped scatter chart presenting the homogeneous regression slopes of the linear relations between pretest and posttest scores by three pedagogies.The third requisite is that the interaction between the independent variable and the covariate (“Pedagogy∗Pretest”) should be insignificant to exclude the interfering effects of the covariate ([46], pp. 208). The test of between-subjects effects revealed that the significance levels of “Pedagogy,” “Pretest scores,” and “Pedagogy∗Pretest” are 0.707, 0.000, and 0.972, respectively. If not set as the covariate, the “Pretest” variable group would exert interfering effects (sig.=0.000<0.05). Notably, the insignificant interaction between “Pedagogy” and “Pretest” (sig.=0.972>0.05) could exclude the interfering effects of “Pretest scores” ([46], pp. 210). Thus, “Data for H1” satisfied the requisite of the insignificant interaction between the independent variable and the covariate.The fourth requisite is the variance equality for the dependent values (“Posttest”) in the treatment groups (“Pedagogy”) ([46], pp. 213). Levene’s test revealed the significance level>0.05 of 0.107, confirming that the “Posttest” variable group did not violate the assumption of the variance equality for all the treatment groups ([46], pp. 213). Thus, the variances of dependent values were relatively equal for all the three treatment groups.Overall, “Data for H1” satisfied all the four requisites for ANCOVA. We conducted ANCOVA to investigate compare the effects on learning achievement for gamification and game-based learning.


## 4.1.2. ANCOVA Comparing the Effects on Learning Achievement for Gamification and Game-Based Learning

ANCOVA reported the average dependent values in all the treatment groups. The descriptive statistics (see Table2) revealed that two experimental groups (N=40, M=79.301, and SD=1.258 for the gamification group; N=40, M=77.473, and SD=1.262 for the game-based learning group) had higher average CET-6 posttest scores than the control group (N=40, M=74.026, and SD=1.258). Specifically speaking, the gamification group (M=79.301) had higher average CET-6 posttest scores than the game-based learning group (M=77.473).Table 2
ANCOVA estimates comparing the average CET-6 posttest scores for the three treatment groups.
Dependent variable: posttest scoresPedagogyMeanStd. error95% confidence intervalLower boundUpper boundGamification79.301a1.25876.81181.792Game-based learning77.473a1.26274.97479.971Traditional didactic74.026a1.25871.53476.518aCovariates appearing in the model are evaluated at the following values: pretestscores=52.5917.ANCOVA also reported whether the average CET-6 posttest scores significantly differed among the three treatment groups based on the covariate of the CET-6 pretest scores. The between-subjects effects test (see Table3) revealed that the average CET-6 posttest scores were significantly different among the three treatment groups (sig.=0.013<0.05). That finding confirmed that game-related pedagogies could significantly influence learning achievement reflected by the average CET-6 posttests.Table 3
The between-subjects effects test examining the significance levels.
Dependent variable: posttest scoresSourceType III sum of squaresdfMean squareFSig.Corrected model8458.939a32819.64644.6430.000Intercept51879.073151879.073821.3950.000Pedagogy573.5722286.7864.5410.013Pretest7906.47217906.472125.1820.000Error7326.52811663.160Total726034.000120Corrected total15785.467119aRsquared=0.536 (adjustedRsquared=0.524).The pairwise comparisons (see Table4) reported the differences between treatment groups in detail. The significant difference in average CET-6 posttest scores between the gamification and control groups (sig.=0.004<0.05) confirmed the significant positive effects of gamification on learning achievement. However, the relatively insignificant difference between the game-based and control groups (sig.=0.056>0.05) suggested that game-based learning exerted insignificantly positive effects on learning achievement. Thus, even if both gamification and game-based learning positively influenced learning achievement reflected by average CET-6 posttest scores, gamification exerted relatively more significant effects on learning achievement than game-based learning.Table 4
Pairwise comparisons examining the significance level for the differences in CET-6 posttest scores for the three treatment groups.
Dependent variablePosttest scores(I) Pedagogy(J) PedagogyMean difference (I-J)Std. errorSig.b95% confidence interval for differencebLower boundUpper boundGamificationGame-based learning1.8291.7850.308-1.7065.363Traditional didactic5.275∗1.7770.0041.7558.795Game-based learningGamification-1.8291.7850.308-5.3631.706Traditional didactic3.4461.7860.056-0.0916.984Traditional didacticGamification-5.275∗1.7770.004-8.795-1.755Game-based learning-3.4461.7860.056-6.9840.091Based on estimated marginal means.∗The mean difference is significant at the 0.05 level. bAdjustment for multiple comparisons: least significant difference (equivalent to no adjustments).Based on these findings, we confirmed H1 that gamification and game-based learning exert differently positive effects on learning achievement. Gamification exerted significantly positive effects on learning achievement, while game-based learning exerted insignificantly positive effects on learning achievement.


## 4.2. Test the Hypothesis that Gamification and Game-Based Learning Exert Differently Positive Effects on Motivation

“Data for H2” involved 108 records from the validly answered questionnaires (37 for the gamification group, 35 for the game-based learning group, and 36 for the control group). We first investigated whether “Data for H2” satisfied reliability, validity, and normal distribution for further analyses.


### 4.2.1. Reliability, Validity, and Normal Distribution that “Data for H2” Satisfied for Further Analyses

Reliability stands for the stability of the result outputs over time ([47], pp. 243). Cronbach’s α is the value to assess the reliability of the data ([47], pp. 251). The overall Cronbach α value is 0.810>0.70 for all the eighteen question items, reflecting overall reliable results ([47], pp. 265). Thus, the data satisfied the demand of reliability.Validity stands for the degree to which the measurements are consistent with the target facets ([48], pp. 14). Factor analysis could reduce the data dimensions by combining variables that approximately assess the same facets ([46], pp. 286). Thus, we performed the factor analysis to examine whether the data could provide valid measurement of the motivation-related subscales.Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) and Bartlett’s Test of Sphericity could identify whether the data are suitable for the factor analysis ([46], pp. 292). The value>0.60 of 0.739 from KMO, the significance level<0.05 of 0.000 from Bartlett’s Test of Sphericity, and the chi-square of 691.547 at df=153 from Bartlett’s Test of Sphericity suggested that we could perform the factor analysis for “Data for H2” ([46], pp. 292).The factor analysis reported the scree plot chart (see Figure4) and the rotated component matrix values (see Table 5). According to the scree plot (see Figure 4), the eigenvalue line steadily decreases from the fifth component and reported four high-loading components standing for the main facets. The rotated component matrix table (see Table 5) demonstrated the correlations between the question items and the four high-loading components ([46], pp. 294). Q1.1-Q1.4, Q2.1-Q2.4, Q3.1-Q3.4, and Q4.1-Q4.6 were most positively correlated with the third, second, fourth, and first components, respectively (see Table 5). These reports confirmed that “Data for H2” involved a valid measurement of motivation-related subscales, satisfying the demand of validity. The first, second, third, and fourth facets were self-efficacy for learning and performance (“Efficacy”), extrinsic goal orientation (“Extrinsic”), intrinsic goal orientation (“Intrinsic”), and control of learning beliefs (“Beliefs”), respectively. Accordingly, we calculated every participant’s average scores for each motivation-related subscale.Figure 4
The scree plot presenting high-loading components for the factor analysis.Table 5
Rotated component matrixa presenting the correlations between question items and high-loading components.
Component1234Q1.10.0510.0620.7790.112Q1.20.0260.0750.716-0.090Q1.30.1250.2310.7770.011Q1.40.1710.0830.7910.002Q2.10.0950.8120.1550.124Q2.20.1030.8640.130-0.034Q2.30.0250.8640.0780.076Q2.40.1540.7890.0990.132Q3.1-0.0620.1440.1240.564Q3.20.0940.126-0.0110.773Q3.30.259-0.032-0.0280.742Q3.40.1100.008-0.0850.767Q4.10.6970.0240.1050.264Q4.20.6610.0430.0660.233Q4.30.7410.1500.0700.061Q4.40.706-0.034-0.032-0.027Q4.50.5920.1820.0500.034Q4.60.7040.0570.194-0.036Extraction method: principal component analysis. Rotation method: Varimax with Kaiser normalization.aRotation converged in 5 iterations.As for normal distribution, we performed the Kolmogorov-Smirnov (K-S) test. We conducted the K-S test without the Lilliefors correction to reduce sensitivity to departures from normality ([46], pp. 254). The test without the Lilliefors correction suggested that “Efficacy,” “Extrinsic,” “Intrinsic,” and “Beliefs” have reached the significant levels>0.05 of 0.123, 0.073, 0.106, and 0.115, respectively. Thus, motivation-related subscale variables satisfied the demand of normal distribution.Thus far, “Data for H2” satisfied the conditions of reliability, validity, and normal distribution. We could conduct ANOVA to compare the effects on motivation for gamification and game-based learning.


### 4.2.2. ANOVA Comparing the Effects on Motivation for Gamification and Game-Based Learning

We first checked the reports about the average scores in the four motivation-related subscales for the three treatment groups (see Table6). Generally speaking, compared with the control group, groups employing game-related pedagogies presented relatively higher average scores in motivation-related subscales. The average score in “Intrinsic” was slightly lower for the game-based learning group than the control group. ANOVA would report whether the average scores in motivation-related subscales were significantly different among the three treatment groups.Table 6
Reports about the average scores in motivation-related subscales for the three treatment groups.
PedagogyEfficacyExtrinsicIntrinsicBeliefsGamificationMean3.69823.75003.85143.4527N37373737Std. deviation0.624650.803640.769330.63694Game-based learningMean3.60002.91433.60713.1643N35353535Std. deviation0.506820.947020.610220.63577Traditional didacticMean3.39812.67363.75692.7361N36363636Std. deviation0.611110.936980.747580.85762TotalMean3.56643.12043.74073.1204N108108108108Std. deviation0.592411.003780.714440.77073The ANOVA results (see Table7) revealed that the average scores in “Extrinsic” and “Beliefs” were significantly different among the three treatment groups due to the significance value<0.05 of 0.000 ([46], pp. 167). However, the average scores in “Efficacy” and “Intrinsic” were insignificantly different among the treatment groups due to the levels>0.05 of 0.088 and 0.348, respectively ([46], pp. 167). Thus, game-related pedagogies would significantly influence participants’ scores in extrinsic goal orientation and control of learning beliefs, whereas game-related pedagogies would insignificantly influence the scores in self-efficacy for learning and performance and intrinsic goal orientation. The relatively lower average score in “Intrinsic” for the game-based learning group than the control group would not mean the counterproductive effects of game-based learning due to the insignificant differences in “Intrinsic” average scores for the three groups.Table 7
ANOVA investigating the effects of game-related approaches on motivation in terms of the four motivation-related subscales.
Sum of squaresdfMean squareFSig.EfficacyBetween groups1.70120.8512.4910.088Within groups35.8511050.341Total37.552107ExtrinsicBetween groups23.340211.67014.5060.000Within groups84.4701050.804Total107.810107IntrinsicBetween groups1.08720.5431.0660.348Within groups53.5291050.510Total54.616107BeliefsBetween groups9.47024.7359.1910.000Within groups54.0911050.515Total63.560107The multiple comparisons (see Table8) reported the differences among the treatment groups in detail and identified which pair(s) of means contributed to the significant value(s) ([46], pp. 163). The difference in average scores in “Efficacy” was significant between the gamification group and the control group (sig.=0.000<0.05), while the difference was insignificant between the game-based learning group and the control group (sig.=0.530>0.05). These findings suggested that gamification would significantly enhance extrinsic goal orientation and that game-based learning would insignificantly enhance extrinsic goal orientation. Besides, the difference in the average scores in “Beliefs” was significant between the gamification group and the control group (sig.=0.000<0.05), and the difference was also significant between the game-based learning group and the control group (sig.=0.047<0.05). These findings suggested that gamification would exert more significant effects on control of learning beliefs despite the overall positive effects on control of learning beliefs for both gamification and game-based learning.Table 8
Multiples comparisons the significance level for the differences in participants’ scoring motivation-related subscales for the three treatment groups.
ScheffeDependent variable(I) Pedagogy(J) PedagogyMean difference (I-J)Std. errorSig.95% confidence intervalLower boundUpper boundEfficacyGamificationGame-based learning0.098200.137780.776-0.24390.4403Traditional didactic0.300050.136790.095-0.03960.6397Game-based learningGamification-0.098200.137780.776-0.44030.2439Traditional didactic0.201850.138710.351-0.14260.5463Traditional didacticGamification-0.300050.136790.095-0.63970.0396Game-based learning-0.201850.138710.351-0.54630.1426ExtrinsicGamificationGame-based learning0.83571∗0.211490.0010.31061.3609Traditional didactic1.07639∗0.209970.0000.55501.5978Game-based learningGamification-0.83571∗0.211490.001-1.3609-0.3106Traditional didactic0.240670.212910.530-0.28800.7694Traditional didacticGamification-1.07639∗0.209970.000-1.5978-0.5550Game-based learning-0.240670.212910.530-0.76940.2880IntrinsicGamificationGame-based learning0.244210.168360.353-0.17380.6623Traditional didactic0.094410.167150.853-0.32060.5095Game-based learningGamification-0.244210.168360.353-0.66230.1738Traditional didactic-0.149800.169490.678-0.57070.2711Traditional didacticGamification-0.094410.167150.853-0.50950.3206Game-based learning0.149800.169490.678-0.27110.5707BeliefsGamificationGame-based learning0.288420.169240.239-0.13180.7086Traditional didactic0.71659∗0.168030.0000.29941.1338Game-based learningGamification-0.288420.169240.239-0.70860.1318Traditional didactic0.42817∗0.170380.0470.00510.8512Traditional didacticGamification-0.71659∗0.168030.000-1.1338-0.2994Game-based learning-0.42817∗0.170380.047-0.8512-0.0051∗The mean difference is significant at the 0.05 level.The test of variance homogeneity (see Table9) reported whether the ANOVA results were reliable and persuasive. The significance levels>0.05 of 0.597, 0.189, 0.518, and 0.063 suggested the homogeneous variances for “Efficacy,” “Extrinsic,” “Intrinsic,” and “Beliefs,” respectively. Thus, the involved variable groups maintained the assumption of the variance equality, confirming the reliability and persuasiveness of the ANOVA results ([46], pp. 213).Table 9
The variance homogeneity test examining the persuasiveness of the ANOVA results.
Levene statisticdf1df2Sig.EfficacyBased on mean0.51921050.597Based on median0.46221050.631Based on median and with adjusteddf0.462299.7520.631Based on trimmed mean.5202105.596ExtrinsicBased on mean1.6942105.189Based on median2.1212105.125Based on median and with adjusteddf2.121298.608.125Based on trimmed mean1.7772105.174IntrinsicBased on mean.6632105.518Based on median.5732105.565Based on median and with adjusteddf.573299.659.566Based on trimmed mean.6482105.525BeliefsBased on mean2.8432105.063Based on median2.9042105.059Based on median and with adjusteddf2.9042102.658.059Based on trimmed mean2.9042105.059Based on these findings, we confirmed H2 that gamification and game-based learning exert differently positive effects on motivation. Compared with game-based learning, gamification exerts more significantly positive effects on extrinsic goal orientation and control of learning beliefs. Gamification and game-based learning exerted insignificant effects on self-efficacy for learning and performance and intrinsic goal orientation. Gamification and game-based learning exerted different effects on motivation reflected by the four motivation-related subscales.


## 4.2.1. Reliability, Validity, and Normal Distribution that “Data for H2” Satisfied for Further Analyses

Reliability stands for the stability of the result outputs over time ([47], pp. 243). Cronbach’s α is the value to assess the reliability of the data ([47], pp. 251). The overall Cronbach α value is 0.810>0.70 for all the eighteen question items, reflecting overall reliable results ([47], pp. 265). Thus, the data satisfied the demand of reliability.Validity stands for the degree to which the measurements are consistent with the target facets ([48], pp. 14). Factor analysis could reduce the data dimensions by combining variables that approximately assess the same facets ([46], pp. 286). Thus, we performed the factor analysis to examine whether the data could provide valid measurement of the motivation-related subscales.Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) and Bartlett’s Test of Sphericity could identify whether the data are suitable for the factor analysis ([46], pp. 292). The value>0.60 of 0.739 from KMO, the significance level<0.05 of 0.000 from Bartlett’s Test of Sphericity, and the chi-square of 691.547 at df=153 from Bartlett’s Test of Sphericity suggested that we could perform the factor analysis for “Data for H2” ([46], pp. 292).The factor analysis reported the scree plot chart (see Figure4) and the rotated component matrix values (see Table 5). According to the scree plot (see Figure 4), the eigenvalue line steadily decreases from the fifth component and reported four high-loading components standing for the main facets. The rotated component matrix table (see Table 5) demonstrated the correlations between the question items and the four high-loading components ([46], pp. 294). Q1.1-Q1.4, Q2.1-Q2.4, Q3.1-Q3.4, and Q4.1-Q4.6 were most positively correlated with the third, second, fourth, and first components, respectively (see Table 5). These reports confirmed that “Data for H2” involved a valid measurement of motivation-related subscales, satisfying the demand of validity. The first, second, third, and fourth facets were self-efficacy for learning and performance (“Efficacy”), extrinsic goal orientation (“Extrinsic”), intrinsic goal orientation (“Intrinsic”), and control of learning beliefs (“Beliefs”), respectively. Accordingly, we calculated every participant’s average scores for each motivation-related subscale.Figure 4
The scree plot presenting high-loading components for the factor analysis.Table 5
Rotated component matrixa presenting the correlations between question items and high-loading components.
Component1234Q1.10.0510.0620.7790.112Q1.20.0260.0750.716-0.090Q1.30.1250.2310.7770.011Q1.40.1710.0830.7910.002Q2.10.0950.8120.1550.124Q2.20.1030.8640.130-0.034Q2.30.0250.8640.0780.076Q2.40.1540.7890.0990.132Q3.1-0.0620.1440.1240.564Q3.20.0940.126-0.0110.773Q3.30.259-0.032-0.0280.742Q3.40.1100.008-0.0850.767Q4.10.6970.0240.1050.264Q4.20.6610.0430.0660.233Q4.30.7410.1500.0700.061Q4.40.706-0.034-0.032-0.027Q4.50.5920.1820.0500.034Q4.60.7040.0570.194-0.036Extraction method: principal component analysis. Rotation method: Varimax with Kaiser normalization.aRotation converged in 5 iterations.As for normal distribution, we performed the Kolmogorov-Smirnov (K-S) test. We conducted the K-S test without the Lilliefors correction to reduce sensitivity to departures from normality ([46], pp. 254). The test without the Lilliefors correction suggested that “Efficacy,” “Extrinsic,” “Intrinsic,” and “Beliefs” have reached the significant levels>0.05 of 0.123, 0.073, 0.106, and 0.115, respectively. Thus, motivation-related subscale variables satisfied the demand of normal distribution.Thus far, “Data for H2” satisfied the conditions of reliability, validity, and normal distribution. We could conduct ANOVA to compare the effects on motivation for gamification and game-based learning.


## 4.2.2. ANOVA Comparing the Effects on Motivation for Gamification and Game-Based Learning

We first checked the reports about the average scores in the four motivation-related subscales for the three treatment groups (see Table6). Generally speaking, compared with the control group, groups employing game-related pedagogies presented relatively higher average scores in motivation-related subscales. The average score in “Intrinsic” was slightly lower for the game-based learning group than the control group. ANOVA would report whether the average scores in motivation-related subscales were significantly different among the three treatment groups.Table 6
Reports about the average scores in motivation-related subscales for the three treatment groups.
PedagogyEfficacyExtrinsicIntrinsicBeliefsGamificationMean3.69823.75003.85143.4527N37373737Std. deviation0.624650.803640.769330.63694Game-based learningMean3.60002.91433.60713.1643N35353535Std. deviation0.506820.947020.610220.63577Traditional didacticMean3.39812.67363.75692.7361N36363636Std. deviation0.611110.936980.747580.85762TotalMean3.56643.12043.74073.1204N108108108108Std. deviation0.592411.003780.714440.77073The ANOVA results (see Table7) revealed that the average scores in “Extrinsic” and “Beliefs” were significantly different among the three treatment groups due to the significance value<0.05 of 0.000 ([46], pp. 167). However, the average scores in “Efficacy” and “Intrinsic” were insignificantly different among the treatment groups due to the levels>0.05 of 0.088 and 0.348, respectively ([46], pp. 167). Thus, game-related pedagogies would significantly influence participants’ scores in extrinsic goal orientation and control of learning beliefs, whereas game-related pedagogies would insignificantly influence the scores in self-efficacy for learning and performance and intrinsic goal orientation. The relatively lower average score in “Intrinsic” for the game-based learning group than the control group would not mean the counterproductive effects of game-based learning due to the insignificant differences in “Intrinsic” average scores for the three groups.Table 7
ANOVA investigating the effects of game-related approaches on motivation in terms of the four motivation-related subscales.
Sum of squaresdfMean squareFSig.EfficacyBetween groups1.70120.8512.4910.088Within groups35.8511050.341Total37.552107ExtrinsicBetween groups23.340211.67014.5060.000Within groups84.4701050.804Total107.810107IntrinsicBetween groups1.08720.5431.0660.348Within groups53.5291050.510Total54.616107BeliefsBetween groups9.47024.7359.1910.000Within groups54.0911050.515Total63.560107The multiple comparisons (see Table8) reported the differences among the treatment groups in detail and identified which pair(s) of means contributed to the significant value(s) ([46], pp. 163). The difference in average scores in “Efficacy” was significant between the gamification group and the control group (sig.=0.000<0.05), while the difference was insignificant between the game-based learning group and the control group (sig.=0.530>0.05). These findings suggested that gamification would significantly enhance extrinsic goal orientation and that game-based learning would insignificantly enhance extrinsic goal orientation. Besides, the difference in the average scores in “Beliefs” was significant between the gamification group and the control group (sig.=0.000<0.05), and the difference was also significant between the game-based learning group and the control group (sig.=0.047<0.05). These findings suggested that gamification would exert more significant effects on control of learning beliefs despite the overall positive effects on control of learning beliefs for both gamification and game-based learning.Table 8
Multiples comparisons the significance level for the differences in participants’ scoring motivation-related subscales for the three treatment groups.
ScheffeDependent variable(I) Pedagogy(J) PedagogyMean difference (I-J)Std. errorSig.95% confidence intervalLower boundUpper boundEfficacyGamificationGame-based learning0.098200.137780.776-0.24390.4403Traditional didactic0.300050.136790.095-0.03960.6397Game-based learningGamification-0.098200.137780.776-0.44030.2439Traditional didactic0.201850.138710.351-0.14260.5463Traditional didacticGamification-0.300050.136790.095-0.63970.0396Game-based learning-0.201850.138710.351-0.54630.1426ExtrinsicGamificationGame-based learning0.83571∗0.211490.0010.31061.3609Traditional didactic1.07639∗0.209970.0000.55501.5978Game-based learningGamification-0.83571∗0.211490.001-1.3609-0.3106Traditional didactic0.240670.212910.530-0.28800.7694Traditional didacticGamification-1.07639∗0.209970.000-1.5978-0.5550Game-based learning-0.240670.212910.530-0.76940.2880IntrinsicGamificationGame-based learning0.244210.168360.353-0.17380.6623Traditional didactic0.094410.167150.853-0.32060.5095Game-based learningGamification-0.244210.168360.353-0.66230.1738Traditional didactic-0.149800.169490.678-0.57070.2711Traditional didacticGamification-0.094410.167150.853-0.50950.3206Game-based learning0.149800.169490.678-0.27110.5707BeliefsGamificationGame-based learning0.288420.169240.239-0.13180.7086Traditional didactic0.71659∗0.168030.0000.29941.1338Game-based learningGamification-0.288420.169240.239-0.70860.1318Traditional didactic0.42817∗0.170380.0470.00510.8512Traditional didacticGamification-0.71659∗0.168030.000-1.1338-0.2994Game-based learning-0.42817∗0.170380.047-0.8512-0.0051∗The mean difference is significant at the 0.05 level.The test of variance homogeneity (see Table9) reported whether the ANOVA results were reliable and persuasive. The significance levels>0.05 of 0.597, 0.189, 0.518, and 0.063 suggested the homogeneous variances for “Efficacy,” “Extrinsic,” “Intrinsic,” and “Beliefs,” respectively. Thus, the involved variable groups maintained the assumption of the variance equality, confirming the reliability and persuasiveness of the ANOVA results ([46], pp. 213).Table 9
The variance homogeneity test examining the persuasiveness of the ANOVA results.
Levene statisticdf1df2Sig.EfficacyBased on mean0.51921050.597Based on median0.46221050.631Based on median and with adjusteddf0.462299.7520.631Based on trimmed mean.5202105.596ExtrinsicBased on mean1.6942105.189Based on median2.1212105.125Based on median and with adjusteddf2.121298.608.125Based on trimmed mean1.7772105.174IntrinsicBased on mean.6632105.518Based on median.5732105.565Based on median and with adjusteddf.573299.659.566Based on trimmed mean.6482105.525BeliefsBased on mean2.8432105.063Based on median2.9042105.059Based on median and with adjusteddf2.9042102.658.059Based on trimmed mean2.9042105.059Based on these findings, we confirmed H2 that gamification and game-based learning exert differently positive effects on motivation. Compared with game-based learning, gamification exerts more significantly positive effects on extrinsic goal orientation and control of learning beliefs. Gamification and game-based learning exerted insignificant effects on self-efficacy for learning and performance and intrinsic goal orientation. Gamification and game-based learning exerted different effects on motivation reflected by the four motivation-related subscales.


## 5. Discussion

This section discussed how the results were significant based on the comparison with the findings from previous studies. We also speculated how stable the effects of game-related pedagogies would be based on the previous studies.The statistical findings revealed that game-related pedagogies could exert positive effects on learning achievement and motivation. As for the effects on learning achievement, games or game-related elements involving playful experiences could inspire individuals to consolidate behaviors and skills associated with academic progress through immersive experiences [4]. Game-related pedagogies also reflected the student-centered learning procedures conducive to information processing ability [31], problem solving capability [15], learning effectiveness [39], and independent learning capability [36]. The literature review demonstrated that the game-related pedagogies could positively influence learning achievement, consistent with the findings from the studies of [9, 23–29, 31].The findings on the positive effects of game-related pedagogies on motivation were also consistent with the findings from the previous studies. Inspiring activities and competitive game mechanics could stimulate individuals’ interest in promoting or preventing particular behaviors to reach targets goals [4, 12]. Vivid presentations of teaching contents from serious games could draw learners’ attention, raise their concentration, and contribute to their acceptance [22, 24]. The findings on the positive effects of game-related pedagogies on motivation were consistent with the findings from the studies of [2, 12, 15, 22, 24, 29, 37–41].Additionally, the positive effects on learning achievement were associated with the enhancement in mental states on game-related pedagogies. As the essential element of game or game-related elements, fun would be the indicator of an optimal state for learning: a balance between challenges and skill levels [4]. When individuals reached the skill levels consistent with the challenge levels of the tasks, individuals tended to enjoy spontaneous and enjoyable participation in the target activities [4]. Based on the balance between challenge and skills, motivated individuals could also enjoy high independency in achieving mastery of the knowledge [2, 15, 30, 32–36]. Thus, game-related pedagogies could contribute to the optimal state where individuals’ learning skills match the difficulty levels of the tasks, and this state is positively associated with learning achievement and motivation. The findings of this study were congruent with the findings of the aforementioned studies.This study also revealed that gamification exerted more significant effects on learning achievement and motivation than game-based learning. This finding was consistent with the statements about the dependence on immersion for game-related pedagogies. Game-related pedagogies emphasize immersive experiences involving the heightened simultaneous experiences of concentration, interest, and enjoyment in the activities [4]. Fun and immersive experiences cultivate learning skills and positive states conducive to learning achievement and motivation. Notably, compared with game-based learning, gamification requires more dependence on immersion because gamification is a set of activities and systematic processes based on the characteristics of game elements rather than the mere use of game incentives such as badges or points [4]. For that reason, gamification creates more immersive experiences associated with learning achievement and motivation, consistent with our finding that gamification would exert more significantly positive effects on learning achievement and motivation.We also speculated how stable the game-related pedagogies would function based on the previous studies. Distraction or frustration caused by external factors may decrease engagement [4, 23], reflecting the relatively unstable effects of game-related pedagogies. Since gamification highly depends on immersion, uncontrollable external factors, e.g., temporal or spatial conditions, or internal factors, e.g., learners’ frustration or distraction, may decrease the efficiency of making progress [4]. Although gamification was more positively associated with engagement conducive to enhanced learning achievement and motivation, gamification would function less stably than game-based learning because gamification highly depends on immersive experiences subject to the uncontrollable factors [4].Overall, our findings on the positive effects of game-related pedagogies on learning achievement and the more positive effects for gamification than game-based learning were congruent with the findings from the previous studies. We also speculated that the effects on learning achievement or motivation would be relatively less stable for gamification than game-based learning due to high dependence on immersion easily influenced by external or internal factors.


## 6. Conclusion

This section demonstrated the answers to the two hypotheses, conclusions from the analyses, and insights for future studies based on potential limitations.


### 6.1. Major Findings

To test H1, we performed ANCOVA to investigate whether gamification and game-based learning exerted differently positive effects on learning achievement. We set the pedagogy types (gamification, game-based learning, and traditional didactic pedagogy) as the independent variable, the CET-6 posttest scores as the dependent variable, and the CET-6 pretest scores as the covariate. “Data for H1” could be suitable for ANCOVA because it satisfied the requisites of normal distribution, homogeneity, insignificant interaction between the independent variable and the covariate, and the variance equality. ANCOVA reported that the average CET-6 posttest score was higher for the gamification group (N=40, M=79.301, and SD=1.258) than the game-based learning group (N=40, M=77.473, and SD=1.262) (see Table 2) and that game-related pedagogies could significantly influence learning achievement (see Table 3). The pairwise comparisons (see Table 4) suggested that the enhancement in learning achievement was more significant for gamification than game-based learning. Thus, we concluded that gamification exerted more significant effects on learning achievement than game-based learning and remained H1 that gamification and game-based learning exert differently positive effects on learning achievement.To test H2, we performed ANOVA to compare the effects on motivation reflected by four motivation-related subscales for gamification and game-based learning. We set the pedagogy types (gamification, game-based learning, and traditional didactic pedagogy) as the independent variable and the scores in the motivation-related subscales (self-efficacy for learning and performance, extrinsic goal orientation, intrinsic goal orientation, and control of learning beliefs) as the dependent variables. “Data for H2” could be suitable for ANOVA because it satisfied the conditions of reliability, validity, and normal distribution. ANOVA (see Table7) reported that the average scores in extrinsic goal orientation and control of learning beliefs were significantly different among the treatment groups. The multiple comparisons (see Table 8) suggested that the positive effects on extrinsic goal orientation and control of learning beliefs were more significant for gamification than game-based learning. The test of variance homogeneity (see Table 9) confirmed the persuasiveness of the results. Thus, we concluded that gamification exerted more significant effects on motivation reflected by the motivation-related subscales than game-based learning and remained H2 that gamification and game-based learning exert differently positive effects on motivation.Based on the tested hypotheses, we concluded that game-related pedagogies could positively influence learning achievement and motivation and that the positive effects on learning achievement and motivation were more significant for gamification than game-based learning. These findings were consistent with the results from the previous studies, reflecting the significance of this study. We also speculated that the high dependence on engagement would influence the stability for the effects of gamification. Gamification and game-based learning would differently function in pedagogical practices because they had distinct features at the theoretical level and different effects on learning achievement and motivation at the empirical level.


### 6.2. Limitations of This Study and Insights for Future Research

Admittedly, this study had some limitations. One limitation might be the coverage of participants. This study involved senior students divided into three treatment groups. We could extend the coverage to pupils, junior high students, senior high students, postgraduates, or doctors. More inclusive coverage of participants could contribute to more comprehensive analyses. Besides, we concentrated on the quantitative assessments of learning achievement and motivation. A combination of quantitative assessments and qualitative procedures, e.g., interview and content analyses, could report more comprehensive results. For more comprehensive empirical designs, we could also conduct the linear regression analyses to identify whether game-related pedagogies could contribute to the stable correlation between learning achievement and motivation.These points could provide the following insights for future research. Future research could investigate the effects of game-related pedagogies on learning achievement or motivation for participants in different grades. Future research could also identify the correlation between learning achievement and motivation through the combination of quantitative and qualitative procedures. The linear relations between learning achievement and motivation could enable researchers to predict how participants’ learning performance and motivation would change in the class employing game-related pedagogies.


## 6.1. Major Findings

To test H1, we performed ANCOVA to investigate whether gamification and game-based learning exerted differently positive effects on learning achievement. We set the pedagogy types (gamification, game-based learning, and traditional didactic pedagogy) as the independent variable, the CET-6 posttest scores as the dependent variable, and the CET-6 pretest scores as the covariate. “Data for H1” could be suitable for ANCOVA because it satisfied the requisites of normal distribution, homogeneity, insignificant interaction between the independent variable and the covariate, and the variance equality. ANCOVA reported that the average CET-6 posttest score was higher for the gamification group (N=40, M=79.301, and SD=1.258) than the game-based learning group (N=40, M=77.473, and SD=1.262) (see Table 2) and that game-related pedagogies could significantly influence learning achievement (see Table 3). The pairwise comparisons (see Table 4) suggested that the enhancement in learning achievement was more significant for gamification than game-based learning. Thus, we concluded that gamification exerted more significant effects on learning achievement than game-based learning and remained H1 that gamification and game-based learning exert differently positive effects on learning achievement.To test H2, we performed ANOVA to compare the effects on motivation reflected by four motivation-related subscales for gamification and game-based learning. We set the pedagogy types (gamification, game-based learning, and traditional didactic pedagogy) as the independent variable and the scores in the motivation-related subscales (self-efficacy for learning and performance, extrinsic goal orientation, intrinsic goal orientation, and control of learning beliefs) as the dependent variables. “Data for H2” could be suitable for ANOVA because it satisfied the conditions of reliability, validity, and normal distribution. ANOVA (see Table7) reported that the average scores in extrinsic goal orientation and control of learning beliefs were significantly different among the treatment groups. The multiple comparisons (see Table 8) suggested that the positive effects on extrinsic goal orientation and control of learning beliefs were more significant for gamification than game-based learning. The test of variance homogeneity (see Table 9) confirmed the persuasiveness of the results. Thus, we concluded that gamification exerted more significant effects on motivation reflected by the motivation-related subscales than game-based learning and remained H2 that gamification and game-based learning exert differently positive effects on motivation.Based on the tested hypotheses, we concluded that game-related pedagogies could positively influence learning achievement and motivation and that the positive effects on learning achievement and motivation were more significant for gamification than game-based learning. These findings were consistent with the results from the previous studies, reflecting the significance of this study. We also speculated that the high dependence on engagement would influence the stability for the effects of gamification. Gamification and game-based learning would differently function in pedagogical practices because they had distinct features at the theoretical level and different effects on learning achievement and motivation at the empirical level.


## 6.2. Limitations of This Study and Insights for Future Research

Admittedly, this study had some limitations. One limitation might be the coverage of participants. This study involved senior students divided into three treatment groups. We could extend the coverage to pupils, junior high students, senior high students, postgraduates, or doctors. More inclusive coverage of participants could contribute to more comprehensive analyses. Besides, we concentrated on the quantitative assessments of learning achievement and motivation. A combination of quantitative assessments and qualitative procedures, e.g., interview and content analyses, could report more comprehensive results. For more comprehensive empirical designs, we could also conduct the linear regression analyses to identify whether game-related pedagogies could contribute to the stable correlation between learning achievement and motivation.These points could provide the following insights for future research. Future research could investigate the effects of game-related pedagogies on learning achievement or motivation for participants in different grades. Future research could also identify the correlation between learning achievement and motivation through the combination of quantitative and qualitative procedures. The linear relations between learning achievement and motivation could enable researchers to predict how participants’ learning performance and motivation would change in the class employing game-related pedagogies.



---

*Source: 9855328-2022-04-23.xml*