Gamification and its assessment in EFL teaching
Creators: name: Novak, Lukas
name: Helvich, Jakub
Publisher: name: OSF Registries
Year: 2022
Subjects: subject: English Language and Literature
subject: Educational Methods
subject: Arts and Humanities
subject: Education
subject: Scholarship of Teaching and Learning
subject: Higher Education
subject: Language and Literacy Education
subject: Secondary Education
subject: EFL
subject: English teaching
subject: MALL
subject: PRISMA
subject: assessment
subject: education
subject: game-based learning
subject: gamification
subject: systematic review
subject: teachers' perceptions
subject: teachers' perspectives
subject: teachers' views

description: Review title:
Gamification and its assessment in EFL teaching

Original language title:
Gamifikace a jeho hodnocení ve výuce anglického jazyka pro nerodilé mluvčí

Keywords:
education, EFL, MALL, English teaching, gamification, game-based learning, assessment, teachers' view, teachers' perception, systematic review, PRISMA

Anticipated or actual start date:
01 May 2022

Anticipated completion date:
01 September 2022

Date of registration in OSF
28 April 2022

Review questions:
Q1: How are teachers satisfied with the applicability of gamification platforms in EFL teaching?
Q2: How do teachers perceive gamification in terms of learning outcomes in EFL teaching?
Q3: How do teachers perceive gamification in terms of pupils' motivation in EFL teaching?

Search strategy:
A unified search terms table was created for all databases used in this systematic review. The final search syntaxes were adjusted to the databases' search interfaces. For Google Scholar two search terms tables were created to overcome its limitation of 32 words (see Searches).

Searches:
We searched for relevant articles in five major databases ERIC, Scopus, WoS, EBSCO, and ProQuest. Considering the sparsity of related systematic reviews, we limited the time range to 2011 – the day of the search in connection to the first gamification summit in 2011 held in San Francisco and Gartner Research of Hype Cycle for Emerging Technologies published in 2011. We chose these databases based on the ranking, broad coverage of studies, and retrieval optimizations. We also included Google Scholar in our systematic review as a secondary source being able to obtain relevant studies outside the selected databases. However, due to its capricious recall and limited search interface, we decided to condition its use in order to increase the reproducibility of the search. Therefore, we included only the first 300 Google Scholar search results. Google Scholar has another serious limitation regarding its search box capacity, which does not allow searches that exceed 32 search words including Boolean operators. Individual databases were searched using the search terms and Boolean operators adjusted to the databases' search interfaces but for Google Scholar a separate search strategy had to be designed. In order to cover as many relevant search terms variations as possible, we used information from our preliminary search and thesaurus.com as our primary resources to scan for synonyms and keywords. Additionally, we included other keywords that are closely related to the subject matter of this systematic review. Moreover, due to the sheer scope of existing gamification platforms, we decided to isolate the most currently prominent gamification platforms and group the others under the prior stated keywords. In the case of Google Scholar, we decided to create a condensed search terms list and conduct 2 searchers. For further information on search terms, tables see: Search strategy

Intervention:
Measuring the effectivity gamification intervention in EFL lessons including the assessment of individual gamification platforms is done through administrated questionnaires or scales given to EFL teachers focusing on their perceptions of gamification in terms of their level of satisfaction regarding the overall applicability of the gamification platforms in EFL lessons as well as the perceived effect of gamification on motivation and learning outcomes of students/pupils.

Types of study to be included:
We included studies conducted at the lower secondary to higher education which corresponds to the ages of pupils/students 11 – 26 coinciding with pre-intermediate to advanced English level (A2 – C1). Additionally, only studies that analysed the effect of digital gamification on Non-native English teaching were included in other words digital gamification of EFL teaching. In the matter of the second group of criteria, we included only peer-reviewed quantitative and mixed studies published in journals or conference proceedings between the years 2011 – the day of the search that were written in English. With respect to the research design, we included studies that exclusively used English questionnaires or scales examining teachers' perspectives.
We excluded studies conducted in preschool and primary education which coincide to pupils younger than 11. We also excluded qualitative or non-peer-reviewed studies that were written in languages other than English or were not focusing on EFL teaching. The systematic review did not include any books, book chapters, other systematic reviews or theses. Furthermore, we excluded studies that did not use English questionnaires or scales or focused on assessments of students', pupils', or parents' perspectives.

Main outcomes:
The primary objectives of this systematic review were to examine the overall satisfaction of teachers with gamification platforms with respect to their applicability in EFL lessons through questionnaires and scale assessments. This includes not only the analysis of the assessment items, comprehensively surveying the extent of teachers' satisfaction but also the conclusions of the respective studies. We examined the items assessing platform applicability in regard to satisfaction with user interface designs as well as in terms of their intrinsic limitations. Moreover, this systematic review aimed to analyse the effects of gamification on pupils' motivation and learning outcomes through EFL teachers' assessments. This also encompasses whether the assessments were based on first-hand experience with a gamification platform or not.

Data extraction:
Abstracts of studies retrieved using our search syntaxes were reviewed by the first author in order to select studies that met our inclusion criteria. The same procedure was repeated by the co-author to decrease the chance of selection bias. If ambiguity accrued, then discussion took place until consensus was reached. A third co-author then made the final decision in cases where discussion did not lead to consensus. The degree of agreement between the authors will be assessed by both percentage of agreement as well as by Cohen’s Kappa in R programming environment via irr package.
The full texts were then reviewed by the first author and assessed for eligibility. The process of study selection is depicted in the PRISMA flow diagram.
After eligibility assessment, the following information will be extracted from the studies: number of participants, educational and English level being gamified, items of assessments measuring teachers’ perceptions of gamification regarding platform applicability, motivation of students and learning outcomes. Furthermore, we will extract other relevant data connected to gamification intervention in EFL lessons, especially in respect to teaching. Lastly, information relevant to our quality measures criteria will be extracted as well. Acquired data will be processed and stored in xlsx file.

Quality assessment:
Studies will be assessed during data extraction for the following quality measures criteria: sample size (the number of EFL teachers’ assessing gamification intervention), reliability of the questionnaires or scales. Moreover, a standardised quality assessment scale will be also used in the process.

Strategy for data synthesis:
Since we decided to include all questionnaires and scales aligning with our objectives the likelihood of them being homogenous is rather slim. Therefore, we will provide synthesis of data not only from the assessments but also from conclusions of respective studies. The resulted synthesis will be presented in a table detailing individual aspect of interests i.e., satisfaction of teachers with gamification regarding the overall applicability and its effect on motivation and learning outcomes of pupils/students. Additionally, we will also include data such as names of selected articles, methodology or other relevant information connected to assessments.

Domain being studied:
Gamification of EFL education. Teachers’ perspectives on gamification.

Contact details:
Jakub Helvich
jakub.helvich@uhk.cz

Organisational affiliation of the review:
Department of Applied Cybernetics, University of Hradec Králové

Review team members and their organisational affiliations:
Lukáš Novák. Social Health Institute of Palacky University Olomouc
Jakub Helvich. Department of Applied Cybernetics, University of Hradec Králové
Štěpán Hubálovský. Department of Applied Cybernetics, University of Hradec Králové

Funding sources/sponsors:
None

Conflicts of interest:
None

Language:
English

Country:
Czech Republic

Stage of review:
Review Ongoing

Type and method of review:
Systematic review, qualitative synthesis of relevant studies

Review stage: Started Completed
Preliminary searches YES YES
Piloting of the study selection process YES YES
Formal screening of search results against eligibility criteria NO NO
Data extraction NO NO
Risk of bias (quality) assessment NO NO
Data analysis NO NO