# Business Simulation Games in Higher Education: A Systematic Review of Empirical Research

Authors: Nadia Faisal; Mehmood Chadhar; Anitra Goriss-Hunter; Andrew Stranieri
Journal: Human Behavior and Emerging Technologies
 (2022)

---

## Abstract

Over the last few years, business simulation games (BSGs) in higher education have attracted attention. BSGs tend to actively engage students with course material, promoting higher engagement and motivation and enabling learning outcomes. Increasingly, researchers are trying to explore the full potential of these games with an upsurge of research in the BSG field in recent years. There is a need to understand the current state of research and future research opportunities; however, there is a lack of recent systematic literature reviews in BSG literature. This study addresses this gap by systematically compiling online empirical research from January 2015 to April 2022. We followed PRISMA guidelines to identify fifty-seven (57) papers reporting empirical evidence of the effectiveness of BSGs in teaching and learning. Findings showed that BSGs improve learning outcomes such as knowledge acquisition, cognitive and interactive skills, and behaviour. The review also summarises different issues concerning the integration of BSGs into the curriculum, learning theories used in the selected studies, and assessment methods used to evaluate student achievement in learning outcomes. The findings of this review summarise the current research activities and indicate existing deficiencies and potential research directions that can be used as the basis for future research into the use of BSGs in higher education.


---

## Body

## 1. Introduction

The simulation and virtual training market was valued at $204.41 billion in 2019 and has been predicted to reach $579.44 billion by the end of 2027 according to the Global Opportunity and Industry Forecast 2020-2027 report [1]. For many years, simulations employing mixed reality have been a sign of the future since they merge several linked disciplines of art and science to generate improved interactive experiences [2]. The cross-border combination of education and gaming has gotten a lot of interest from many parts of society, including education and technology. By providing a fun and engaging learning environment and encouraging students to learn via hands-on experience, simulation games have the potential to enhance students’ advanced skills [3]. Business simulations were established in the 1950s, and the simulation industry has grown significantly since then, with a significant growth in usage by business schools in higher education institutions. With the availability of off-the-shelf and customised business simulation games (BSGs), universities started integrating them into their courses in the mid-60s to provide an active learning experience to the students [4]. Since then, business simulations have been increasingly used to enhance students’ learning. This development has resulted in an increase in research in this field. Academics have attempted to establish the advantages of these games via evidence-based research [5]. Research evidence is used to inform policymakers about the effectiveness of a specific educational approach. However, single studies present contextual and methodological limitations of research evidence as most empirical research are done within a confined context [6]. Therefore, there is a need to identify, evaluate, and synthesise research results from different empirical studies focusing on a single phenomenon to create a summary of current evidence. If done systematically, a literature review provides researchers and practitioners with a broad overview of the research area, shows what work has been done, helps identify research gaps, and directs future research in that particular subject area [7].Past reviews summarising the impact of using BSGs as part of pedagogical approaches to meet learning outcomes show ambiguous results and lack empirical evidence. Furthermore, the extensive effort and timescale involved in carrying out and then publishing literature reviews make them obsolete relatively quickly. By the time they are completed, many new studies have been done on the same phenomena with different or sometimes conflicting results [8]. Fu et al. [9] conducted the last literature review on the reported empirical evidence of the positive impact of BSGs on learning, summarising studies from 2005 to 2014. However, a significant expansion has been observed in BSG research, as shown in Figure 1.Figure 1
No. of publications in BSG literature from Jan 2015 to Feb 2022.Some of the newer games, teaching and evaluation techniques, and positive effects of these games on different learning and behavioural outcomes were examined and explored during this period. There is a value in systematically compiling these studies to provide a sound basis for further endeavours in this field. Therefore, this review is aimed at presenting a systematic literature review (SLR) of empirical studies on BSGs published between January 2015 and February 2022. The review addresses the research question: “What empirical evidence exists concerning the use of BSGs in promoting learning and effective teaching?” During the literature review, four distinct themes emerged that were then used to create four subquestions to analyse the fifty-seven selected papers for this systematic literature review. Figure2 represents the themes.
(i)
Integration of BSGs in higher education: this theme addresses the integration of business simulation games to teach IT and business courses. This section also describes the instructors’ role in facilitating the learning process by using BSGs and the barriers to integrating these games(ii)
Learning theories/models and BSGs: different learning theories and models are identified under this theme in BSG research, and links between these theories/models and simulation games are established(iii)
Evaluation of BSGs: this theme discusses frameworks and guidelines to efficiently evaluate the simulation games’ learning outcomes(iv)
Learning outcomes of BSGs: this theme addresses the different learning outcomes that can be achieved using simulation games as a learning tool. The learning outcomes are categorised into skills, knowledge, and behavioural effectsFigure 2
Themes in BSG literature.Overall, this SLR provides an overview of empirical research, what is currently being investigated, and possible future directions in BSGs research.The structure of the rest of the paper is as follows. Section2 presents the background of BSGs and discusses the past reviews on BSGs. Section 3 describes the research methodology. The results are shown in Section 4. Section 5 discusses the findings of the study. The limitations and future recommendations of the selected papers are discussed in Section 6, and Section 7 concludes the paper with a brief discussion of future research agendas.


## 2. Background

In this section, a brief introduction to BSGs, previous reviews, and their limitations are presented.


### 2.1. Business Simulation Games (BSGs)

BSGs are experiential learning tools where students learn business processes by running a simulated firm in a risk-free, interactive, and realistic environment. Students make all strategic decisions and compete either individually or in teams. These active learning experiences improve student engagement and develop collaboration, decision-making, problem-solving, and critical thinking [4]. BSGs have been very popular in business education since their inception. With the technological developments in operations research, war games, computer technology, and education theory, educators started integrating these games into business courses in the late 1950s [10]. Games such as the Beer Distribution Game [11] and the Markstrat Simulation Game [12] were released in the following years and gained popularity among educators.Technology developments such as interactive features, advanced graphics processing, artificial intelligence, and cloud computing have influenced business simulations [13, 14]). Furthermore, BSGs are sometimes used as incentives to make crowdsourcing (crowd-based online work) more valuable and attractive [15]. Global enterprises like Google, Microsoft, American Express, and Caterpillar are using these simulation games to train their employees and managers [16]. Currently, many business courses include one or more management simulations. They are more commonly used at the undergraduate level to provide an active learning experience to students [17]. In addition to encouraging higher engagement levels, simulations in the business context can improve soft skills such as teamwork, decision-making, leadership, and other technical skills used in strategic management, marketing, finance, and project management.In contrast to passive learning experiences that can occur in traditional teaching techniques, such as lectures and tutorials, BSGs bridge the gap between academia and industry through experiential learning techniques. In BSGs, students run a simulated company to learn business processes and strategies. They get the opportunity to integrate and experiment with what they have learned through simulations, solve complex problems, get involved in active decision-making, experience the consequences of their decisions, and learn from their mistakes [18]. BSGs are an effective teaching technique to help students develop managerial and generic skills in demand in the industry [19].


### 2.2. Past Reviews on Simulation Games

Several studies summarised the literature on simulation games in the past ten years, but most reviews target a specific and limited feature of simulation games. Table1 presents some of the reviews and meta-analyses published between 2009 and 2022 on game-based learning and their respective scopes:Table 1
Previous literature reviews on simulation games.
Year and authorFocus of reviewFerreira et al. [20]SLR on research related to the learning process with (serious) business games using data collection techniques with electroencephalogram or eye tracking signalsStanitsas et al. [21]Simulation games facilitate sustainable education. The primary focus was on sustainabilitySubhash and Cudney [22]Categorisation of gamified and game-based learningSubhash and Cudney [22]Evaluation methods in computing educationBoyle et al. [23]Reported empirical evidence of the positive impact of computer games on learningFu, Hainey and Baxter [9]Reported empirical evidence of the effect of BSGs on learning outcomesGiessen [24]Serious games’ role in improving learning outcomesHamari, Koivisto and Sarsa [25]Reported empirical studies on the effect of gamification on learningLopes et al. [26]Reported effect of business games on leadership developmentWouters [27]Reported effect of business games on leadership developmentConnolly [28]Reported empirical evidence of the positive impact of computer games on learningTobias [29]Reported empirical evidence of the use of serious games in learningSitzmann [30]Industrial and organisational psychology and managementJahangirian et al. [31]Review simulation applications within manufacturing and business fieldsAlthough the past reviews focused on the simulation games’ effectiveness in building knowledge and skills, there are limitations in these studies. For example, the latest SLR by Ferreira et al. [20] only focused on researchers that used electroencephalogram (EEG) or eye tracking (ET) signals as data collection methods. Another limitation of this review is the limited amount of research linking ET and EEG devices to help study the BSG user experience which is a drawback of this work. These devices have employed marketing, human behaviour, and applied neuroscience research. However, they are integrated with other methodologies, comparing and contrasting findings. This paper lacks methodological integration, leading to skewed outcomes. In another review, Sitzmann [30] statistically summarised 52 studies on simulation games’ effectiveness in enhancing industry-related knowledge and skills. The review focused on adult work competency requirements in organisational settings and did not discuss simulation games in educational settings. The study findings showed that technology could improve the learning experience and stressed the importance of multiple factors (integration, training, and debriefing) which can contribute to rich learning experiences through gamified learning. A review by Connolly et al. [28] addressed the impact of games on overall learning outcomes. That review had a broad scope, and instead of just focusing on simulation games, it considered all computer games (entertainment games, serious games, video games, simulation games, etc.). It concluded that computer games positively impact students’ cognitive, affective, behavioural, and motivational outcomes. The most significant gains were in knowledge acquisition and affective and motivational outcomes. In 2016, the same researchers [23] presented an updated version of their previous review, including entertainment and educational computer games. This review again demonstrated positive impact on learning outcomes but did not specifically focus on computer simulation games. Lopes et al. [26] summarised the effect of using simulation games as a means of meeting learning outcomes, but again, the review only covered leadership development through simulation games. Scholtz and Hughes [32] conducted a systematic literature review to explore the innovative and new pedagogical methods instructors use to integrate simulation games into business courses. The scope of that review was limited to exploring the educators’ role in the learning process.


## 2.1. Business Simulation Games (BSGs)

BSGs are experiential learning tools where students learn business processes by running a simulated firm in a risk-free, interactive, and realistic environment. Students make all strategic decisions and compete either individually or in teams. These active learning experiences improve student engagement and develop collaboration, decision-making, problem-solving, and critical thinking [4]. BSGs have been very popular in business education since their inception. With the technological developments in operations research, war games, computer technology, and education theory, educators started integrating these games into business courses in the late 1950s [10]. Games such as the Beer Distribution Game [11] and the Markstrat Simulation Game [12] were released in the following years and gained popularity among educators.Technology developments such as interactive features, advanced graphics processing, artificial intelligence, and cloud computing have influenced business simulations [13, 14]). Furthermore, BSGs are sometimes used as incentives to make crowdsourcing (crowd-based online work) more valuable and attractive [15]. Global enterprises like Google, Microsoft, American Express, and Caterpillar are using these simulation games to train their employees and managers [16]. Currently, many business courses include one or more management simulations. They are more commonly used at the undergraduate level to provide an active learning experience to students [17]. In addition to encouraging higher engagement levels, simulations in the business context can improve soft skills such as teamwork, decision-making, leadership, and other technical skills used in strategic management, marketing, finance, and project management.In contrast to passive learning experiences that can occur in traditional teaching techniques, such as lectures and tutorials, BSGs bridge the gap between academia and industry through experiential learning techniques. In BSGs, students run a simulated company to learn business processes and strategies. They get the opportunity to integrate and experiment with what they have learned through simulations, solve complex problems, get involved in active decision-making, experience the consequences of their decisions, and learn from their mistakes [18]. BSGs are an effective teaching technique to help students develop managerial and generic skills in demand in the industry [19].


## 2.2. Past Reviews on Simulation Games

Several studies summarised the literature on simulation games in the past ten years, but most reviews target a specific and limited feature of simulation games. Table1 presents some of the reviews and meta-analyses published between 2009 and 2022 on game-based learning and their respective scopes:Table 1
Previous literature reviews on simulation games.
Year and authorFocus of reviewFerreira et al. [20]SLR on research related to the learning process with (serious) business games using data collection techniques with electroencephalogram or eye tracking signalsStanitsas et al. [21]Simulation games facilitate sustainable education. The primary focus was on sustainabilitySubhash and Cudney [22]Categorisation of gamified and game-based learningSubhash and Cudney [22]Evaluation methods in computing educationBoyle et al. [23]Reported empirical evidence of the positive impact of computer games on learningFu, Hainey and Baxter [9]Reported empirical evidence of the effect of BSGs on learning outcomesGiessen [24]Serious games’ role in improving learning outcomesHamari, Koivisto and Sarsa [25]Reported empirical studies on the effect of gamification on learningLopes et al. [26]Reported effect of business games on leadership developmentWouters [27]Reported effect of business games on leadership developmentConnolly [28]Reported empirical evidence of the positive impact of computer games on learningTobias [29]Reported empirical evidence of the use of serious games in learningSitzmann [30]Industrial and organisational psychology and managementJahangirian et al. [31]Review simulation applications within manufacturing and business fieldsAlthough the past reviews focused on the simulation games’ effectiveness in building knowledge and skills, there are limitations in these studies. For example, the latest SLR by Ferreira et al. [20] only focused on researchers that used electroencephalogram (EEG) or eye tracking (ET) signals as data collection methods. Another limitation of this review is the limited amount of research linking ET and EEG devices to help study the BSG user experience which is a drawback of this work. These devices have employed marketing, human behaviour, and applied neuroscience research. However, they are integrated with other methodologies, comparing and contrasting findings. This paper lacks methodological integration, leading to skewed outcomes. In another review, Sitzmann [30] statistically summarised 52 studies on simulation games’ effectiveness in enhancing industry-related knowledge and skills. The review focused on adult work competency requirements in organisational settings and did not discuss simulation games in educational settings. The study findings showed that technology could improve the learning experience and stressed the importance of multiple factors (integration, training, and debriefing) which can contribute to rich learning experiences through gamified learning. A review by Connolly et al. [28] addressed the impact of games on overall learning outcomes. That review had a broad scope, and instead of just focusing on simulation games, it considered all computer games (entertainment games, serious games, video games, simulation games, etc.). It concluded that computer games positively impact students’ cognitive, affective, behavioural, and motivational outcomes. The most significant gains were in knowledge acquisition and affective and motivational outcomes. In 2016, the same researchers [23] presented an updated version of their previous review, including entertainment and educational computer games. This review again demonstrated positive impact on learning outcomes but did not specifically focus on computer simulation games. Lopes et al. [26] summarised the effect of using simulation games as a means of meeting learning outcomes, but again, the review only covered leadership development through simulation games. Scholtz and Hughes [32] conducted a systematic literature review to explore the innovative and new pedagogical methods instructors use to integrate simulation games into business courses. The scope of that review was limited to exploring the educators’ role in the learning process.


## 3. Research Methodology

A systematic literature review (SLR) helps to compile relevant research on a particular phenomenon of interest or topic [33]. In line with Keele [34] guidelines, this study used a systematic process to conduct a comprehensive literature review. This process involves developing protocols for the literature search, identifying and selecting relevant primary studies, extracting and synthesising data, and reporting results.The broad objective of this SLR was to answer the following research question.RQ: “What empirical evidence exists concerning using BSGs to promote learning and effective teaching?”The subquestions that emerged from the coding of the literature were as follows:(i)
What are the learning outcomes of BSGs?(ii)
How are BSGs integrated into existing curricula to support learning outcomes?(iii)
What were the learning theories used in BSG literature?(iv)
What assessment methods are used in BSG literature to evaluate learning outcomes and game performance?Learning outcomes are classified into behavioural, knowledge, and skill outcomes [35].


### 3.1. Data Sources and Search Strings

Databases used to search for the relevant open access research papers were EBSCOhost, Emerald insight, IEEE, Informit, JSTOR, Oxford, Sage, ScienceDirect, Scopus, SpringerLink, Tylor & Francis, Web of Science, Wiley Online Library, and Google Scholar. These databases are the best and most supportive resources for computing game research.Initially, we conducted Boolean searches using the “AND” operator between the keywords business simulation games, learning outcomes, and higher education. To avoid losing relevant data, researchers also used alternative words by conducting a Boolean search again, but they used “OR” as an operator this time. Table2 shows the main key terms and alternative search terms for data search.Table 2
Terms used in the paper search.
Key search termsAlternate search termsBusiness simulation gamesSerious games, computer games in business education, game-based learning in businessLearning outcomesLearning outcomes, skills, behaviours, attitudesHigher educationBusiness school, universities, vocational trainingThe snowballing method [36] was applied to exhaust the included papers’ relevant sources to minimise any data loss. This process was considered complete when no pertinent new papers were found. The literature search was last updated on 28th February 2022.


### 3.2. Selection Process

This review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews (PRISMA) guidelines. Frequently, systematic reviews indicate a lack of comprehension of common principles that enable them to be replicable and scientifically competent. PRISMA is a common peer-reviewed methodology that utilises a checklist of guidelines, which was carefully adhered to in this work. It adds to the revision process’s quality assurance and reproducibility. We created a protocol outlining the article selection criteria, search technique, data extraction, and data analysis methods. The researchers followed the Dybå and Dingsøyr [37] citation management process to sort out the relevant citations with the help of EndNote 20. The complete selection process is shown in Figure 3.Figure 3
Paper selection process.


#### 3.2.1. Inclusion Criteria

Papers that fulfil the following criteria were included:(i)
Only peer-reviewed journal and conference papers were included(ii)
Only empirical papers that reported on research using data collected from experiments or observation designs were included(iii)
The participants should be studying in higher education institutes (graduate and postgraduate business courses)(iv)
The review included only open access articles published between 2015 and 2022(v)
The papers were available in English(vi)
The papers explicitly report the learning outcomes of BSGs


#### 3.2.2. Exclusion Criteria

(i)
Nonempirical studies are not included in this review(ii)
Studies conducted with participants younger than 18 are not included as the focus is on higher education institutions(iii)
Studies on nondigital and entertainment games are not included in this review(iv)
Book chapters are not included as they are difficult to search on databases and hard to access as full texts. The peer-review process for journal and conference papers does not always apply to books. Other sources were also excluded for the same reason. For example, dissertations, theses, editorials, book reviews, and reports are also excluded for similar reasons(v)
Papers that are not open access are not included(vi)
Papers that do not adhere to the research objectives are not included(vii)
Papers published before January 2015 are also excludedCoauthors of this SLR reviewed the selected papers to ensure they met the inclusion criteria. In cases of doubt, all four reviewers discussed and determined the inclusion or exclusion decision on the paper.


### 3.3. Quality Assessment

In the third stage, a quality assessment of the included papers was conducted. Extra discussions among researchers were undertaken to ensure the objectivity and quality of the selected papers. The quality assessment criteria gave each of the 57 selected papers on a five-dimensional scale inspired by Connolly et al. [28]. Five dimensions were scored from 1 to 3 across each dimension, where 3 represented high, 2 medium, and 1 low quality of the paper. In this paper, we adopted the following five dimensions in Table 3.Table 3
Quality assessment criteria.
Quality assessment criteriaIndicators (1-3)Is the research focusing on BSGs?Low-medium-highIs the research design suitable to address the objectives of this SLR?Low-medium-highIs the focus of the study relevant to the objective of this SLR?Low-medium-highIs the methodology of the study well defined and well structured?Low-medium-highAre the findings of the study answering the research questions?Low-medium-highThe total quality weight of each paper was determined by adding scores for all five dimensions ranging from 5 to 15. The mean rating was 8.9, and the mode rating was 9. 37 papers reaching 9 or above scores on the five dimensions scale were considered methodologically more critical papers, providing strong evidence of the impact of BSGs on learning outcomes. These papers are shown in Table3 with their respective methodological designs.


### 3.4. Data Analysis and Synthesis

The 57 selected papers were analysed using a qualitative content analysis, which involved coding for the emergence of themes. The researchers examined the papers from a range of viewpoints. Firstly, they conducted a bibliographic overview which included demographics, research design, year of publication, database of publications, and citation counts of the selected papers. Secondly, the papers were examined to determine the learning outcomes, learning theory, integration techniques, and assessment methods that were discussed in BSG research. The categorisation is aimed at helping instructors select the most highly ranked games according to the required learning outcomes, the evaluation methods used to assess these outcomes, integration of the games into the curriculum, and the role of instructors in facilitating the learning.


## 3.1. Data Sources and Search Strings

Databases used to search for the relevant open access research papers were EBSCOhost, Emerald insight, IEEE, Informit, JSTOR, Oxford, Sage, ScienceDirect, Scopus, SpringerLink, Tylor & Francis, Web of Science, Wiley Online Library, and Google Scholar. These databases are the best and most supportive resources for computing game research.Initially, we conducted Boolean searches using the “AND” operator between the keywords business simulation games, learning outcomes, and higher education. To avoid losing relevant data, researchers also used alternative words by conducting a Boolean search again, but they used “OR” as an operator this time. Table2 shows the main key terms and alternative search terms for data search.Table 2
Terms used in the paper search.
Key search termsAlternate search termsBusiness simulation gamesSerious games, computer games in business education, game-based learning in businessLearning outcomesLearning outcomes, skills, behaviours, attitudesHigher educationBusiness school, universities, vocational trainingThe snowballing method [36] was applied to exhaust the included papers’ relevant sources to minimise any data loss. This process was considered complete when no pertinent new papers were found. The literature search was last updated on 28th February 2022.


## 3.2. Selection Process

This review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews (PRISMA) guidelines. Frequently, systematic reviews indicate a lack of comprehension of common principles that enable them to be replicable and scientifically competent. PRISMA is a common peer-reviewed methodology that utilises a checklist of guidelines, which was carefully adhered to in this work. It adds to the revision process’s quality assurance and reproducibility. We created a protocol outlining the article selection criteria, search technique, data extraction, and data analysis methods. The researchers followed the Dybå and Dingsøyr [37] citation management process to sort out the relevant citations with the help of EndNote 20. The complete selection process is shown in Figure 3.Figure 3
Paper selection process.


### 3.2.1. Inclusion Criteria

Papers that fulfil the following criteria were included:(i)
Only peer-reviewed journal and conference papers were included(ii)
Only empirical papers that reported on research using data collected from experiments or observation designs were included(iii)
The participants should be studying in higher education institutes (graduate and postgraduate business courses)(iv)
The review included only open access articles published between 2015 and 2022(v)
The papers were available in English(vi)
The papers explicitly report the learning outcomes of BSGs


### 3.2.2. Exclusion Criteria

(i)
Nonempirical studies are not included in this review(ii)
Studies conducted with participants younger than 18 are not included as the focus is on higher education institutions(iii)
Studies on nondigital and entertainment games are not included in this review(iv)
Book chapters are not included as they are difficult to search on databases and hard to access as full texts. The peer-review process for journal and conference papers does not always apply to books. Other sources were also excluded for the same reason. For example, dissertations, theses, editorials, book reviews, and reports are also excluded for similar reasons(v)
Papers that are not open access are not included(vi)
Papers that do not adhere to the research objectives are not included(vii)
Papers published before January 2015 are also excludedCoauthors of this SLR reviewed the selected papers to ensure they met the inclusion criteria. In cases of doubt, all four reviewers discussed and determined the inclusion or exclusion decision on the paper.


## 3.2.1. Inclusion Criteria

Papers that fulfil the following criteria were included:(i)
Only peer-reviewed journal and conference papers were included(ii)
Only empirical papers that reported on research using data collected from experiments or observation designs were included(iii)
The participants should be studying in higher education institutes (graduate and postgraduate business courses)(iv)
The review included only open access articles published between 2015 and 2022(v)
The papers were available in English(vi)
The papers explicitly report the learning outcomes of BSGs


## 3.2.2. Exclusion Criteria

(i)
Nonempirical studies are not included in this review(ii)
Studies conducted with participants younger than 18 are not included as the focus is on higher education institutions(iii)
Studies on nondigital and entertainment games are not included in this review(iv)
Book chapters are not included as they are difficult to search on databases and hard to access as full texts. The peer-review process for journal and conference papers does not always apply to books. Other sources were also excluded for the same reason. For example, dissertations, theses, editorials, book reviews, and reports are also excluded for similar reasons(v)
Papers that are not open access are not included(vi)
Papers that do not adhere to the research objectives are not included(vii)
Papers published before January 2015 are also excludedCoauthors of this SLR reviewed the selected papers to ensure they met the inclusion criteria. In cases of doubt, all four reviewers discussed and determined the inclusion or exclusion decision on the paper.


## 3.3. Quality Assessment

In the third stage, a quality assessment of the included papers was conducted. Extra discussions among researchers were undertaken to ensure the objectivity and quality of the selected papers. The quality assessment criteria gave each of the 57 selected papers on a five-dimensional scale inspired by Connolly et al. [28]. Five dimensions were scored from 1 to 3 across each dimension, where 3 represented high, 2 medium, and 1 low quality of the paper. In this paper, we adopted the following five dimensions in Table 3.Table 3
Quality assessment criteria.
Quality assessment criteriaIndicators (1-3)Is the research focusing on BSGs?Low-medium-highIs the research design suitable to address the objectives of this SLR?Low-medium-highIs the focus of the study relevant to the objective of this SLR?Low-medium-highIs the methodology of the study well defined and well structured?Low-medium-highAre the findings of the study answering the research questions?Low-medium-highThe total quality weight of each paper was determined by adding scores for all five dimensions ranging from 5 to 15. The mean rating was 8.9, and the mode rating was 9. 37 papers reaching 9 or above scores on the five dimensions scale were considered methodologically more critical papers, providing strong evidence of the impact of BSGs on learning outcomes. These papers are shown in Table3 with their respective methodological designs.


## 3.4. Data Analysis and Synthesis

The 57 selected papers were analysed using a qualitative content analysis, which involved coding for the emergence of themes. The researchers examined the papers from a range of viewpoints. Firstly, they conducted a bibliographic overview which included demographics, research design, year of publication, database of publications, and citation counts of the selected papers. Secondly, the papers were examined to determine the learning outcomes, learning theory, integration techniques, and assessment methods that were discussed in BSG research. The categorisation is aimed at helping instructors select the most highly ranked games according to the required learning outcomes, the evaluation methods used to assess these outcomes, integration of the games into the curriculum, and the role of instructors in facilitating the learning.


## 4. Results

This section presents results from the selected papers (n=57).


### 4.1. Bibliographic Overview of the Papers

The included papers are journal publications except two [38, 39]. The included papers presented multiple study designs, with most papers falling into survey design. Most papers are retrieved from the SCOPUS database, followed by ScienceDirect. The number of papers per year and per data base is shown in Figures 4 and 5, respectively. The papers have an average citation count of 10±20 (till February 2022).Figure 4
Number of selected papers per year.Figure 5
Number of selected papers per database.Due to cultural settings and available resources, demographic differences can cause varying attitudes and expectations toward learning [40]. The number of papers published in the region also shows its willingness to adapt and experiment with innovative knowledge and teaching methods and tools. Most of the studies on simulation games (Figure 6) were conducted in Europe (n=33) followed by North America (n=10). Asia (n=8) comes after them, followed by Australia, Africa, and the Middle East.Figure 6
Number of papers as per region.


### 4.2. Methodologies of Selected Studies

There are different types of constructs and variables used in the selected papers. Most empirical studies on BSGs evaluated how playing the game might assist students in meeting learning outcomes. Entrepreneurial skills and attitudes are other commonly used constructs in the BSG literature. A range of study designs (quantitative, qualitative, and mixed methods) was used in the selected papers depending on the research objectives. Figure7 presents an overview of the study design.Figure 7
No. of papers per study design.Most of the papers used postgame surveys (questionnaires) to collect data (28 papers), and the participants were students in the selected studies except three papers. Mainly, informal methods were used to develop these questionnaires without defining a measurement model or theoretical construct. 13 studies used tests to assess the students’ learning outcomes and skills acquisition, typically in experimental studies and by applying them before and after playing the game. However, a few studies conducted surveys just after the game session (8 studies). Five studies reported the use of challenge or exercise for data collection. Interviews with students on how playing the BSGs helped them to achieve learning outcomes were also used, typically in qualitative design or mixed-method approaches. These interviews were done most of the time after playing the game. Only 3 studies included interviews with instructors. In five studies, researchers conducted focus group meetings to collect opinions on the effectiveness of BSGs in enhancing students’ learning experiences. Figure8 shows the number of papers as per the data collection method used in the study.Figure 8
No. of papers as per data collection method.BSGs are usually played in teams, and most studies conducted on BSGs collected data over large samples (100-500), usually in postgame surveys or game results. Three selected studies used extensive samples, with one analysing 3681 messages posted by students on an online discussion forum [41], one analysing 3129 game results [39], and one longitudinal study surveying more than 12000 students over three years [42]. The number of papers as per sample size range is shown in Figure 9.Figure 9
No. of papers as per sample size.Quantitative data were mostly analysed through statistical analysis using different software. Regression analysis,t-test, and one-way ANOVA are the most commonly employed tests on the quantitative data sets in selected studies. A comprehensive bibliographic review of the selected papers is given in Tables 4 and 5.Table 4
Bibliographical overview of the selected papers.
PaperDatabaseCitationPublication typeStudy designStudy focus/contributionVos [43]EBSCOhost87Journal paperExploratoryAuthentic assessment strategyChen, Keys and Gaber [44]EBSCOhost15Journal paperConfirmatoryEffect of enjoyment and cognitive appraisal on learning outcomesBurdon, Munro [45]EBSCOhost19Journal paperCase studyExperiences of design, development, and implementation of simulationEder, Antonucci and Monk [46]EBSCOhostJournal paperExamine the association of learning outcomes with student engagement and team dynamicsBitrián, Buil and Catalán [47]EBSCOhost3Journal paperQuasiexperimentalEffect of flow on learning outcomesGhani, Mohammad [48]EBSCOhost1Journal paperConfirmatoryAimed to find the integration of the logic model that contributes to effective entrepreneurial learningThanasi-Boçe [49]EmeraldJournal paperExploratoryEffect on entrepreneurial mindsetTawil et al. [39]Google Scholar11Conference paperEntrepreneur skills in decision-makingÕun, Mägi and Noppel [50]Google Scholar1Journal paperComparative analysisEffects of personality and cultural difference on learning outcomes using BSGLovelace, Eggers and Dyck [51]Google Scholar67Journal paperExploratoryDevelopment of critical thinking through simulation and its relationship with game performanceAlmeida [52]Google Scholar13Journal paperLongitudinal studyEntrepreneurship learning through business simulationsCostin, O’Brien and Hynes [19]Google Scholar25Journal paperCase studyDevelopment of entrepreneurial skillsMohsen, Abdollahi and Omar [53]Google Scholar11Journal paperEducational values generated from a SGUrquidi-Martín, Tamarit-Aznar and Sánchez-García [54]Google Scholar7Journal paperConfirmatoryDevelop critical thinking focused on sustainabilityAlmeida and Buzady [55]Google Scholar2Journal paperExploratoryDevelopment of entrepreneurship competenciesBuzady and Almeida [56]Google Scholar7Conference paperExploratoryDevelopment of management, leadership, and entrepreneurship skillsBach, Zoroja and Fašnik [17]Google Scholar2Conference paperConfirmatoryInvestigate the level of usage of simulation games at faculties of economics as compared to other types of teachingZulfiqar et al. [57]Google Scholar14Journal paperLongitudinal studyCompared the impact of traditional teaching and teaching through online management simulation games on student learning performance and further leads to entrepreneurial intentionYusof [58]Google Scholar0Journal paperConfirmatorySeeks to link the effectiveness of business simulation with entrepreneurship interest of the studentsLovin et al. [59]Google Scholar7Journal paperConfirmatoryExamined the roles of graduates’ engagement in business simulations, working environment culture, and acquired knowledge on business simulations in knowledge transferDharmastuti et al. [60]Google Scholar1Journal paperConfirmatoryExamine how a business simulation class student’s experience impacts student business competency and learning outcomes in learning business simulationFaisal et al. [61]Google Scholar3Conference papersExploratoryExplored the effects of ERPsim game on the work readiness of IS graduatesBeranič and Heričko [62]Google Scholar2Journal paperConfirmatoryInvestigating impact of business simulation on knowledge acquisition and future student engagementHishiyama and Nakajima [63]IEEEConference proceedingCase studyManagement flow functionsTao, Yeh and Hung [64]JSTOR15Journal paperExperimentalPerceived learning-cycle effects caused by playing multiple BSGs with different complexityEllahi, Zaka and Sultan [65]JSTOR12Journal paperExperimentalAnalysing success of supplementary components for the existing teaching strategiesWilliams [66]Sage10Journal paperActionDevelopment of entrepreneurial capabilities through simulationBeuk [67]Sage12Journal paperCross-sectionalComparing three teaching methods (lectures, case studies, and simulation) in terms of learning outcomesKriz, Auchter [42]Sage38Journal paperLongitudinal studyIncrease in entrepreneurial competencies a result of startup simulation courses and cup competitionsZulfiqar [68]Sage31Journal paperConfirmatoryInfluence of simulation on attitude and intentions of the students toward entrepreneurial activitiesRogmans and Abaza [69]Sage6Journal paperComparative analysisAnalysing students’ engagement levels in two different teaching methods, simulation, and case studyWang et al. [70]Sage1Journal paperInvestigated the factors that influence students’ BSG usage intentionKiss and Schmuck [71]Sage3Journal paperLongitudinalInvestigated the influence of games on managerial skillsKuang, Adler and Pandey [72]Sage2Journal paperQuasiexperimentalPositive effect on higher order thinking skillsGoi [4]SpringerLink26Journal paperSequential explanatoryRole of authentic team based learning in enhancing the learning outcomes and satisfactionLevant, Coulmont and Sandu [73]Tylor & Francis47Journal paperDevelopment of soft skills under the impact of cultural backgroundHernández-Lara and Serradell-López [41]Tylor & Francis11Journal paperExploratoryEducational effectiveness of business simulation games based on the students’ opinionsHumpherys, Bakir and Babb [74]Tylor & Francis4Journal paperConfirmatoryInvestigated simulation as an experiential learning toolObi, Eze and Chibuzo [75]Tylor & Francis2Journal paperConfirmatoryDetermining the experiential learning activities required of business education students for the development of various 21st century competenciesBuil, Catalán and Martínez [76]Wiley4Journal paperConfirmatoryFacilitate students’ learning and engagementOlive et al. [77]ScienceDirect1Journal paperCarenys, Moya and Perramon [78]ScienceDirect33Jourtal paperExperimentalEffectiveness of videogames in comparison to simulationsLeal-Rodriguez and Albort-Morant [79]ScienceDirect38Journal paperImpact on academic performanceȘtefan et al. [80]ScienceDirect0Journal paperExamine the effect of simulation on collaborative decision-making skillsSeverengiz [38]ScienceDirect1Conference paperEffect of simulation on factory planning knowledgeLoon, Evans and Kerridge [81]Scopus65Journal paperMultimethodRole of instructional design and the context of the study in strategic decision-makingBell and Loon [82]Scopus67Journal paperQuasiexperimentalDetermine whether learning through business simulations is affected by students’ critical thinking dispositionLee, Long and Visinescu [83]Scopus4Journal paperQuasiexperimentalDevelopment of business intelligence through BSGPando-Garcia, Periañez-Cañadillas and Charterina [84]Scopus62Journal paperQuasiexperimentalConstructs of ease of use and perceived usefulness and their effects on attitude to use and intention to use between two groups of students using different modesLin, Yen and Wang [85]Scopus19Journal paperExperimentalEffect of learning method and motivation on learning performance in BSGsHwang, Cruthirds [86]Scopus13Journal paperExperimentalComparison of online and traditional teaching methods using BSGS on ERP, SAP, and business process knowledgeUrquidi Martín and Tamarit Aznar [87]Scopus6Journal paperQuasiexperimentalEmpirical evidence on the effectiveness of business simulations in university teachingMustata, et al. [88]Scopus16Journal paperCase studyDevelopment of management-related competenciesNewbery, et al. [89]Scopus47Journal paperQuasiexperimentalImpact of the initial entrepreneurial experience on identity formation using BSGTorres and Augusto [90]Scopus29Journal paperImpact of BSG on students’ self-perception of their improvement on strategic competenciesFarashahi and Tajeddin [91]Scopus54Journal paperComparative analysisComparison of learning outcomes of lectures, case study, and simulationAlas et al. [92]ScopusJournal paperQuasiexperimentalThe relationship between the results obtained by different teams in business game Dynama and their teams’ characteristicsCalabor, Mora and Moya [93]Scopus20Journal paperDelphiAccounting academic perceptions of the usefulness and the potential barriers to implementing BSGsZulfiqar et al. [57]Scopus10Journal paperConfirmatoryCompared the impact of traditional teaching and teaching through online management simulation games on student learning performance and entrepreneurial intentionBeranič and Heričko [62]Scopus0Journal paperConfirmatoryInvestigating simulation impact on significant knowledge gain and students’ future course engagementSamaras, Adkins and White [94]Scopus2Journal paperConfirmatoryCompared case studies and simulations to provide insights into how each may contribute to the development and demonstration of students’ critical thinking skillsTable 5
Quality ratings and methodologies of selected papers.
PaperQuality ratingData collectionSample and instrumentsData analysisConstructs used in the studyVos [43]11Mixed methodsPostgame 35 surveys and 8 interviews from studentsQualitativeChen, Keys, and Gaber [44]12QuantitativePostgame surveys from 164 studentsSPSS partial least squares (pls)Behavioural intentions, enjoyment, cognitive appraisal, perceived learning outcomesBurdon and Munro[45]13QualitativeQualitative surveys from studentsThematic analysisEder, Antonucci, and Monk [46]12QuantitativePostgame surveys from 118 studentsStatistical analysisEngagement, team dynamics, learning outcomesBitrián, Buil, and Catalán [47]9QuantitativePre-postgame surveys from 430 studentsSPSS two cluster analysisStudents’ perceived learning, skills, and satisfaction, boredom, flow, anxiety, apathyGhani and Mohammad [48]13QuantitativePre-postgame surveys from 272 studentsConfirmatory factor analysisStudent characteristics, lecturer characteristics, simulation characteristics, business plan learning effectivenessThanasi-Boçe [49]QualitativePostgame 16 open-ended surveys from studentsThematic analysisTawil et al. [39]9Quantitative3129 students game resultsÕun, Mägi, and Noppel [50]9QuantitativePostgame survey from 118 students from four countriesLovelace, Eggers, and Dyck [51]13QuantitativePre-postgame surveys from 98 studentsSPSS paired samplet-test and descriptive analysisCritical thinking, problem-solving, game performanceAlmeida [52]12QuantitativePostgame surveys from 83 studentsStata software v13.0 descriptive analysisTechnical competencies, management skills, personal entrepreneurshipCostin, O’Brien, and Hynes [19]14QualitativeReflective essaysThematic analysisDecision-making, problem-solving, communication and teamwork, risk managementMohsen, Abdollahi, and Omar [53]14Mixed methodsPostgame surveys and reflection reports of 120 studentsSPSS exploratory factor analysis and reliability analysis, thematic analysisStudents’ experience generation, conceptual understanding, skills development, and affective evaluation, respectivelyUrquidi-Martín, Tamarit-Aznar, and Sánchez-García [54]13QuantitativePostgame surveys from 326 studentsAnalysis of the causal relationshipsGame realism, game structure, perceived usefulness, motivation, critical thinkingAlmeida and Buzady [55]Mixed methodsPostgame focus group discussion and game resultsDescriptive and thematicGame performance and 29 management and leadership skillsBuzady and Almeida [56]13Mixed methodsPostgame surveys and interviews from 52 studentsDescriptive and thematicIndividual attitudes, 29 MAP dimensionsBach, Zoroja, and Fašnik [17]13QuantitativePostgame surveysDescriptive analysis and chi square testAdvantages of different types of teaching methodsZulfiqar et al. [57]14QuantitativePostgame survey from 277 studentsStructural equation model (SEM)Perceived ease of use and perceived usefulness, knowledge application and knowledge sharing, learnability, self-efficacy and perceived enjoyment, technology adoption and learning performance, entrepreneurial intentionsYusof [58]13QuantitativePostgame survey from 160 studentsMultiple regression analysisLearning, benefit, satisfaction, and perception of business simulationLovin et al. [59]14QuantitativePostgame survey from 120 graduatesMultiple regression analysisKnowledge transfer, engagement, working environment culture, acquired knowledge from business simulationDharmastuti et al. [60]13QuantitativePostgame survey from 83 studentsDescriptive analysisPerceived usefulness, perceived ease, perceived enjoyment, student business competency, perceived learning outcomesFaisal et al. [61]14QualitativeInterviews from 15 instructorsThematic analysisLearning outcomes, behavioural changes, work readinessBeranič and Heričko [62]10QuantitativePre-postgame surveyDescriptive analysisBusiness processes knowledge, technical knowledge of SAP, ERP transaction knowledge, intent for future course engagementHishiyama and Nakajima [63]10Mixed method138 students and 180 instructorsDescriptive, ANVOVATao, Yeh, and Hung [64]13QuantitativePostgame surveys from 43 studentsStatistical analysis, descriptive, ANVOVA, paired samplet-testComplexity level, skills, declarative knowledge, procedural knowledge, strategic knowledge, matching the competitionEllahi, Zaka, and Sultan [65]13QuantitativePre-postsurvey from 87 studentsSPSS, independent samplet-testLearning satisfaction, learning performance, learner’s interestWilliams [66]14Mixed methodsPre-postgame survey from 32 students+reflection reports and game logsIndependent samplet-test and thematic analysisEntrepreneurial skills, entrepreneurial attitudes, business skillsBeuk [67]13QuantitativePostgame surveys from 137 students, 248 instructorsRepeated measures one-way ANVOVA, regression analysisPerceived usefulness, level of fun, instructors perceived learning outcomesKriz and Auchter [42]12QuantitativePostgame surveys from 12521 studentsDescriptiveOrganisation and facilitation of the simulation game, personal and social skills, competition and teamwork, business knowledge and entrepreneurship skills, overall satisfactionZulfiqar et al. [68]13QuantitativePostgame 360 students surveyStructural equation modelling (Garris et al.) using AMOS 24Perceived business value, subjective norms, perceived behavioural controls, attitude toward entrepreneurship and entrepreneurial intentionsRogmans and Abaza [69]11QuantitativePostgame 200 students surveyDescriptiveMotivation and engagementWang et al. [70]14QuantitativePostgame surveys from 141 studentsPartial least square approach SmartPLS softwarePerformance expectancy, effort expectancy, social influence, facilitating conditions, hedonic motivation, and price value, behavioural intentions to use business simulation gamesKiss and Schmuck [71]13QuantitativePostgame survey from 329 studentsFrequency tables, univariate ANCOVA,t-tests and cross-tabulationStrategy formulation, planning, decision-making, and teamwork; mathematical financial skills, managerial skillsGoi [4]14Mixed methodPostgame 365 students’ surveys and 14 students focus groupExploratory factor analysis (SPSS) and confirmatory factor analysis (AMOS) for quantitative and thematic analysis for qualitativeTeamwork, learning outcomes, satisfactionLevant, Coulmont, and Sandu [73]13Mixed methodsPre-postsurveys from 392 students, mi- of Likert scale and open-ended questionsEmployability skills, soft skillsHernández-Lara and Serradell-López [41]14Qualitative3681 messages posted in discussion forumIn vivo data analysisHumpherys, Bakir, and Babb [74]10QuantitativeComparison of project grades of studentsDescriptive analysisObi, Eze, and Chibuzo [75]12QuantitativePostgame survey from 210 studentsCronbach’s alpha reliabilityCommunication and collaboration competencies, critical thinking competencies, academic instructionBuil, Catalán, and Martínez [76]13QuantitativePostgame surveys from 360 studentsStatistical analysisCompetence, autonomy, relatedness, self-efficacy, cognitive engagement, emotional engagement, behavioural engagement, skill development, and perceived learningOlive et al. [77]10Mixed methods114 students pre-post achievement tests, feedback surveys and analysis of trace filesStatistical analysisCarenys, Moya, and Perramon [78]12QuantitativePostgame surveys from 132 studentsExploratory factor analysis and Cronbach’s alpha test of reliabilityAttributes, motivation, and cognitive learning outcomesLeal-Rodriguez and Albort-Morant [79]13Quantitative80 students end of semester grades and game resultsPearson correlation and structural equations modellingStudents performance and students learningȘtefan et al. [80]9QualitativeNot mentionedNot mentionedSeverengiz, Seliger, and Krüger [38]13Postgame surveys from 31 studentsDescriptive studyLoon, Evans, and Kerridge [81]14Mixed methodsPostgame surveys 155 and 36 semistructured interviews from studentsPearson correlation and multiple regression analysis SPSSBell and Loon [82]14QuantitativePostgame surveys from 173 studentsPrincipal component analysis, correlation and regression analysisEngagement, cognitive maturity, innovativeness, intended learning outcomesLee, Long, and Visinescu [83]13QuantitativePostgame surveys from 93 studentsPartial least square approachActive learning, meaningful learning, collaboration, subject integrationPando-Garcia, Periañez-Cañadillas, and Charterina [84]11QuantitativePostgame surveys from two groups, 131 online, 83 onsite studentsConfirmatory factor analysisPerceived usefulness, perceived ease of use (PEOU), attitude to the business game technology, intention to use a business game technologyLin, Yen, and Wang [85]10QuantitativeExperimental scenarios, an achievement test and a motivation scale, 49 response from individual and 47 response from groups of studentsDescriptive analysis, two-way ANOVA on SPSSLearning methods (collaborative and individual), learning motivation, learning performanceHwang and Cruthirds [86]12QuantitativePre-postgame surveys from 52 from studentsDescriptive analysisSAP ease of use, business process knowledge, enterprise system knowledge, sap transaction knowledgeUrquidi Martín, and Tamarit Aznar [87]10QuantitativePostgame survey from 58 studentsDescriptive analysisEvaluation of the experience, learning, and development of critical thinkingMustata et al. [88]10QualitativePostgame qualitative surveys from 88 studentsThematic analysisNewbery et al. [89]10QuantitativePre-postgame surveys from 236 students divided in treatment and control groupst-test and regressionGroup level micro identity, individual level micro identity, and interpersonal micro identity, observed entrepreneurial behaviour, experienced entrepreneurial behavioursTorres and Augusto [90]12QuantitativePostgame surveys from 22 MBA executive studentsANOVAStrategy formulation, strategy implementation, critical analysisFarashahi and Tajeddin [91]14QuantitativePostgame surveys from 194 undergraduate and MBA studentsDescriptive analysisProblem-solving skills, interpersonal skills, and self-awarenessAlas et al. [92]9Quantitative28 game dataOn SPSS, correlation analyses, regression analyses, andt-testsGPA, expenditure of market information, point for homework, normalized profitsCalabor, Mora, and Moya [93]14Quantitative12 academics DelphiDescriptive analysisTechnical aspects of game, learning values of games, general view of gameZulfiqar et al. [57]12QuantitativeTime-lagged surveys from 277 studentsStructural equation modellingKnowledge sharing, knowledge application, learnability, perceived pleasure, and self-efficacyBeranič and Heričko [62]12QuantitativePre-postgame survey from 32 students involved in ERPsim introductory simulationDescriptive analysisBusiness process knowledge, technical SAP knowledge, ERP transaction knowledge, intent for future course engagementSamaras, Adkins, and White [94]13QuantitativePostsurvey from 119 studentsPaired-samplet-testsCritical thinking process and simulation participation


### 4.3. Results of the Quality Assessment

The researchers selected the papers based on quality ratings. The minimum acceptable number was 9, and the maximum number given on the paper was 15. The average quality rating of the selected papers is 11. A standard deviation of 1.7 was found in the quality ratings for the total mean score. As shown in Figure10, there has been a steady increase in the quality of empirical papers on BSGs over time.Figure 10
Mean quality rating per year.


## 4.1. Bibliographic Overview of the Papers

The included papers are journal publications except two [38, 39]. The included papers presented multiple study designs, with most papers falling into survey design. Most papers are retrieved from the SCOPUS database, followed by ScienceDirect. The number of papers per year and per data base is shown in Figures 4 and 5, respectively. The papers have an average citation count of 10±20 (till February 2022).Figure 4
Number of selected papers per year.Figure 5
Number of selected papers per database.Due to cultural settings and available resources, demographic differences can cause varying attitudes and expectations toward learning [40]. The number of papers published in the region also shows its willingness to adapt and experiment with innovative knowledge and teaching methods and tools. Most of the studies on simulation games (Figure 6) were conducted in Europe (n=33) followed by North America (n=10). Asia (n=8) comes after them, followed by Australia, Africa, and the Middle East.Figure 6
Number of papers as per region.


## 4.2. Methodologies of Selected Studies

There are different types of constructs and variables used in the selected papers. Most empirical studies on BSGs evaluated how playing the game might assist students in meeting learning outcomes. Entrepreneurial skills and attitudes are other commonly used constructs in the BSG literature. A range of study designs (quantitative, qualitative, and mixed methods) was used in the selected papers depending on the research objectives. Figure7 presents an overview of the study design.Figure 7
No. of papers per study design.Most of the papers used postgame surveys (questionnaires) to collect data (28 papers), and the participants were students in the selected studies except three papers. Mainly, informal methods were used to develop these questionnaires without defining a measurement model or theoretical construct. 13 studies used tests to assess the students’ learning outcomes and skills acquisition, typically in experimental studies and by applying them before and after playing the game. However, a few studies conducted surveys just after the game session (8 studies). Five studies reported the use of challenge or exercise for data collection. Interviews with students on how playing the BSGs helped them to achieve learning outcomes were also used, typically in qualitative design or mixed-method approaches. These interviews were done most of the time after playing the game. Only 3 studies included interviews with instructors. In five studies, researchers conducted focus group meetings to collect opinions on the effectiveness of BSGs in enhancing students’ learning experiences. Figure8 shows the number of papers as per the data collection method used in the study.Figure 8
No. of papers as per data collection method.BSGs are usually played in teams, and most studies conducted on BSGs collected data over large samples (100-500), usually in postgame surveys or game results. Three selected studies used extensive samples, with one analysing 3681 messages posted by students on an online discussion forum [41], one analysing 3129 game results [39], and one longitudinal study surveying more than 12000 students over three years [42]. The number of papers as per sample size range is shown in Figure 9.Figure 9
No. of papers as per sample size.Quantitative data were mostly analysed through statistical analysis using different software. Regression analysis,t-test, and one-way ANOVA are the most commonly employed tests on the quantitative data sets in selected studies. A comprehensive bibliographic review of the selected papers is given in Tables 4 and 5.Table 4
Bibliographical overview of the selected papers.
PaperDatabaseCitationPublication typeStudy designStudy focus/contributionVos [43]EBSCOhost87Journal paperExploratoryAuthentic assessment strategyChen, Keys and Gaber [44]EBSCOhost15Journal paperConfirmatoryEffect of enjoyment and cognitive appraisal on learning outcomesBurdon, Munro [45]EBSCOhost19Journal paperCase studyExperiences of design, development, and implementation of simulationEder, Antonucci and Monk [46]EBSCOhostJournal paperExamine the association of learning outcomes with student engagement and team dynamicsBitrián, Buil and Catalán [47]EBSCOhost3Journal paperQuasiexperimentalEffect of flow on learning outcomesGhani, Mohammad [48]EBSCOhost1Journal paperConfirmatoryAimed to find the integration of the logic model that contributes to effective entrepreneurial learningThanasi-Boçe [49]EmeraldJournal paperExploratoryEffect on entrepreneurial mindsetTawil et al. [39]Google Scholar11Conference paperEntrepreneur skills in decision-makingÕun, Mägi and Noppel [50]Google Scholar1Journal paperComparative analysisEffects of personality and cultural difference on learning outcomes using BSGLovelace, Eggers and Dyck [51]Google Scholar67Journal paperExploratoryDevelopment of critical thinking through simulation and its relationship with game performanceAlmeida [52]Google Scholar13Journal paperLongitudinal studyEntrepreneurship learning through business simulationsCostin, O’Brien and Hynes [19]Google Scholar25Journal paperCase studyDevelopment of entrepreneurial skillsMohsen, Abdollahi and Omar [53]Google Scholar11Journal paperEducational values generated from a SGUrquidi-Martín, Tamarit-Aznar and Sánchez-García [54]Google Scholar7Journal paperConfirmatoryDevelop critical thinking focused on sustainabilityAlmeida and Buzady [55]Google Scholar2Journal paperExploratoryDevelopment of entrepreneurship competenciesBuzady and Almeida [56]Google Scholar7Conference paperExploratoryDevelopment of management, leadership, and entrepreneurship skillsBach, Zoroja and Fašnik [17]Google Scholar2Conference paperConfirmatoryInvestigate the level of usage of simulation games at faculties of economics as compared to other types of teachingZulfiqar et al. [57]Google Scholar14Journal paperLongitudinal studyCompared the impact of traditional teaching and teaching through online management simulation games on student learning performance and further leads to entrepreneurial intentionYusof [58]Google Scholar0Journal paperConfirmatorySeeks to link the effectiveness of business simulation with entrepreneurship interest of the studentsLovin et al. [59]Google Scholar7Journal paperConfirmatoryExamined the roles of graduates’ engagement in business simulations, working environment culture, and acquired knowledge on business simulations in knowledge transferDharmastuti et al. [60]Google Scholar1Journal paperConfirmatoryExamine how a business simulation class student’s experience impacts student business competency and learning outcomes in learning business simulationFaisal et al. [61]Google Scholar3Conference papersExploratoryExplored the effects of ERPsim game on the work readiness of IS graduatesBeranič and Heričko [62]Google Scholar2Journal paperConfirmatoryInvestigating impact of business simulation on knowledge acquisition and future student engagementHishiyama and Nakajima [63]IEEEConference proceedingCase studyManagement flow functionsTao, Yeh and Hung [64]JSTOR15Journal paperExperimentalPerceived learning-cycle effects caused by playing multiple BSGs with different complexityEllahi, Zaka and Sultan [65]JSTOR12Journal paperExperimentalAnalysing success of supplementary components for the existing teaching strategiesWilliams [66]Sage10Journal paperActionDevelopment of entrepreneurial capabilities through simulationBeuk [67]Sage12Journal paperCross-sectionalComparing three teaching methods (lectures, case studies, and simulation) in terms of learning outcomesKriz, Auchter [42]Sage38Journal paperLongitudinal studyIncrease in entrepreneurial competencies a result of startup simulation courses and cup competitionsZulfiqar [68]Sage31Journal paperConfirmatoryInfluence of simulation on attitude and intentions of the students toward entrepreneurial activitiesRogmans and Abaza [69]Sage6Journal paperComparative analysisAnalysing students’ engagement levels in two different teaching methods, simulation, and case studyWang et al. [70]Sage1Journal paperInvestigated the factors that influence students’ BSG usage intentionKiss and Schmuck [71]Sage3Journal paperLongitudinalInvestigated the influence of games on managerial skillsKuang, Adler and Pandey [72]Sage2Journal paperQuasiexperimentalPositive effect on higher order thinking skillsGoi [4]SpringerLink26Journal paperSequential explanatoryRole of authentic team based learning in enhancing the learning outcomes and satisfactionLevant, Coulmont and Sandu [73]Tylor & Francis47Journal paperDevelopment of soft skills under the impact of cultural backgroundHernández-Lara and Serradell-López [41]Tylor & Francis11Journal paperExploratoryEducational effectiveness of business simulation games based on the students’ opinionsHumpherys, Bakir and Babb [74]Tylor & Francis4Journal paperConfirmatoryInvestigated simulation as an experiential learning toolObi, Eze and Chibuzo [75]Tylor & Francis2Journal paperConfirmatoryDetermining the experiential learning activities required of business education students for the development of various 21st century competenciesBuil, Catalán and Martínez [76]Wiley4Journal paperConfirmatoryFacilitate students’ learning and engagementOlive et al. [77]ScienceDirect1Journal paperCarenys, Moya and Perramon [78]ScienceDirect33Jourtal paperExperimentalEffectiveness of videogames in comparison to simulationsLeal-Rodriguez and Albort-Morant [79]ScienceDirect38Journal paperImpact on academic performanceȘtefan et al. [80]ScienceDirect0Journal paperExamine the effect of simulation on collaborative decision-making skillsSeverengiz [38]ScienceDirect1Conference paperEffect of simulation on factory planning knowledgeLoon, Evans and Kerridge [81]Scopus65Journal paperMultimethodRole of instructional design and the context of the study in strategic decision-makingBell and Loon [82]Scopus67Journal paperQuasiexperimentalDetermine whether learning through business simulations is affected by students’ critical thinking dispositionLee, Long and Visinescu [83]Scopus4Journal paperQuasiexperimentalDevelopment of business intelligence through BSGPando-Garcia, Periañez-Cañadillas and Charterina [84]Scopus62Journal paperQuasiexperimentalConstructs of ease of use and perceived usefulness and their effects on attitude to use and intention to use between two groups of students using different modesLin, Yen and Wang [85]Scopus19Journal paperExperimentalEffect of learning method and motivation on learning performance in BSGsHwang, Cruthirds [86]Scopus13Journal paperExperimentalComparison of online and traditional teaching methods using BSGS on ERP, SAP, and business process knowledgeUrquidi Martín and Tamarit Aznar [87]Scopus6Journal paperQuasiexperimentalEmpirical evidence on the effectiveness of business simulations in university teachingMustata, et al. [88]Scopus16Journal paperCase studyDevelopment of management-related competenciesNewbery, et al. [89]Scopus47Journal paperQuasiexperimentalImpact of the initial entrepreneurial experience on identity formation using BSGTorres and Augusto [90]Scopus29Journal paperImpact of BSG on students’ self-perception of their improvement on strategic competenciesFarashahi and Tajeddin [91]Scopus54Journal paperComparative analysisComparison of learning outcomes of lectures, case study, and simulationAlas et al. [92]ScopusJournal paperQuasiexperimentalThe relationship between the results obtained by different teams in business game Dynama and their teams’ characteristicsCalabor, Mora and Moya [93]Scopus20Journal paperDelphiAccounting academic perceptions of the usefulness and the potential barriers to implementing BSGsZulfiqar et al. [57]Scopus10Journal paperConfirmatoryCompared the impact of traditional teaching and teaching through online management simulation games on student learning performance and entrepreneurial intentionBeranič and Heričko [62]Scopus0Journal paperConfirmatoryInvestigating simulation impact on significant knowledge gain and students’ future course engagementSamaras, Adkins and White [94]Scopus2Journal paperConfirmatoryCompared case studies and simulations to provide insights into how each may contribute to the development and demonstration of students’ critical thinking skillsTable 5
Quality ratings and methodologies of selected papers.
PaperQuality ratingData collectionSample and instrumentsData analysisConstructs used in the studyVos [43]11Mixed methodsPostgame 35 surveys and 8 interviews from studentsQualitativeChen, Keys, and Gaber [44]12QuantitativePostgame surveys from 164 studentsSPSS partial least squares (pls)Behavioural intentions, enjoyment, cognitive appraisal, perceived learning outcomesBurdon and Munro[45]13QualitativeQualitative surveys from studentsThematic analysisEder, Antonucci, and Monk [46]12QuantitativePostgame surveys from 118 studentsStatistical analysisEngagement, team dynamics, learning outcomesBitrián, Buil, and Catalán [47]9QuantitativePre-postgame surveys from 430 studentsSPSS two cluster analysisStudents’ perceived learning, skills, and satisfaction, boredom, flow, anxiety, apathyGhani and Mohammad [48]13QuantitativePre-postgame surveys from 272 studentsConfirmatory factor analysisStudent characteristics, lecturer characteristics, simulation characteristics, business plan learning effectivenessThanasi-Boçe [49]QualitativePostgame 16 open-ended surveys from studentsThematic analysisTawil et al. [39]9Quantitative3129 students game resultsÕun, Mägi, and Noppel [50]9QuantitativePostgame survey from 118 students from four countriesLovelace, Eggers, and Dyck [51]13QuantitativePre-postgame surveys from 98 studentsSPSS paired samplet-test and descriptive analysisCritical thinking, problem-solving, game performanceAlmeida [52]12QuantitativePostgame surveys from 83 studentsStata software v13.0 descriptive analysisTechnical competencies, management skills, personal entrepreneurshipCostin, O’Brien, and Hynes [19]14QualitativeReflective essaysThematic analysisDecision-making, problem-solving, communication and teamwork, risk managementMohsen, Abdollahi, and Omar [53]14Mixed methodsPostgame surveys and reflection reports of 120 studentsSPSS exploratory factor analysis and reliability analysis, thematic analysisStudents’ experience generation, conceptual understanding, skills development, and affective evaluation, respectivelyUrquidi-Martín, Tamarit-Aznar, and Sánchez-García [54]13QuantitativePostgame surveys from 326 studentsAnalysis of the causal relationshipsGame realism, game structure, perceived usefulness, motivation, critical thinkingAlmeida and Buzady [55]Mixed methodsPostgame focus group discussion and game resultsDescriptive and thematicGame performance and 29 management and leadership skillsBuzady and Almeida [56]13Mixed methodsPostgame surveys and interviews from 52 studentsDescriptive and thematicIndividual attitudes, 29 MAP dimensionsBach, Zoroja, and Fašnik [17]13QuantitativePostgame surveysDescriptive analysis and chi square testAdvantages of different types of teaching methodsZulfiqar et al. [57]14QuantitativePostgame survey from 277 studentsStructural equation model (SEM)Perceived ease of use and perceived usefulness, knowledge application and knowledge sharing, learnability, self-efficacy and perceived enjoyment, technology adoption and learning performance, entrepreneurial intentionsYusof [58]13QuantitativePostgame survey from 160 studentsMultiple regression analysisLearning, benefit, satisfaction, and perception of business simulationLovin et al. [59]14QuantitativePostgame survey from 120 graduatesMultiple regression analysisKnowledge transfer, engagement, working environment culture, acquired knowledge from business simulationDharmastuti et al. [60]13QuantitativePostgame survey from 83 studentsDescriptive analysisPerceived usefulness, perceived ease, perceived enjoyment, student business competency, perceived learning outcomesFaisal et al. [61]14QualitativeInterviews from 15 instructorsThematic analysisLearning outcomes, behavioural changes, work readinessBeranič and Heričko [62]10QuantitativePre-postgame surveyDescriptive analysisBusiness processes knowledge, technical knowledge of SAP, ERP transaction knowledge, intent for future course engagementHishiyama and Nakajima [63]10Mixed method138 students and 180 instructorsDescriptive, ANVOVATao, Yeh, and Hung [64]13QuantitativePostgame surveys from 43 studentsStatistical analysis, descriptive, ANVOVA, paired samplet-testComplexity level, skills, declarative knowledge, procedural knowledge, strategic knowledge, matching the competitionEllahi, Zaka, and Sultan [65]13QuantitativePre-postsurvey from 87 studentsSPSS, independent samplet-testLearning satisfaction, learning performance, learner’s interestWilliams [66]14Mixed methodsPre-postgame survey from 32 students+reflection reports and game logsIndependent samplet-test and thematic analysisEntrepreneurial skills, entrepreneurial attitudes, business skillsBeuk [67]13QuantitativePostgame surveys from 137 students, 248 instructorsRepeated measures one-way ANVOVA, regression analysisPerceived usefulness, level of fun, instructors perceived learning outcomesKriz and Auchter [42]12QuantitativePostgame surveys from 12521 studentsDescriptiveOrganisation and facilitation of the simulation game, personal and social skills, competition and teamwork, business knowledge and entrepreneurship skills, overall satisfactionZulfiqar et al. [68]13QuantitativePostgame 360 students surveyStructural equation modelling (Garris et al.) using AMOS 24Perceived business value, subjective norms, perceived behavioural controls, attitude toward entrepreneurship and entrepreneurial intentionsRogmans and Abaza [69]11QuantitativePostgame 200 students surveyDescriptiveMotivation and engagementWang et al. [70]14QuantitativePostgame surveys from 141 studentsPartial least square approach SmartPLS softwarePerformance expectancy, effort expectancy, social influence, facilitating conditions, hedonic motivation, and price value, behavioural intentions to use business simulation gamesKiss and Schmuck [71]13QuantitativePostgame survey from 329 studentsFrequency tables, univariate ANCOVA,t-tests and cross-tabulationStrategy formulation, planning, decision-making, and teamwork; mathematical financial skills, managerial skillsGoi [4]14Mixed methodPostgame 365 students’ surveys and 14 students focus groupExploratory factor analysis (SPSS) and confirmatory factor analysis (AMOS) for quantitative and thematic analysis for qualitativeTeamwork, learning outcomes, satisfactionLevant, Coulmont, and Sandu [73]13Mixed methodsPre-postsurveys from 392 students, mi- of Likert scale and open-ended questionsEmployability skills, soft skillsHernández-Lara and Serradell-López [41]14Qualitative3681 messages posted in discussion forumIn vivo data analysisHumpherys, Bakir, and Babb [74]10QuantitativeComparison of project grades of studentsDescriptive analysisObi, Eze, and Chibuzo [75]12QuantitativePostgame survey from 210 studentsCronbach’s alpha reliabilityCommunication and collaboration competencies, critical thinking competencies, academic instructionBuil, Catalán, and Martínez [76]13QuantitativePostgame surveys from 360 studentsStatistical analysisCompetence, autonomy, relatedness, self-efficacy, cognitive engagement, emotional engagement, behavioural engagement, skill development, and perceived learningOlive et al. [77]10Mixed methods114 students pre-post achievement tests, feedback surveys and analysis of trace filesStatistical analysisCarenys, Moya, and Perramon [78]12QuantitativePostgame surveys from 132 studentsExploratory factor analysis and Cronbach’s alpha test of reliabilityAttributes, motivation, and cognitive learning outcomesLeal-Rodriguez and Albort-Morant [79]13Quantitative80 students end of semester grades and game resultsPearson correlation and structural equations modellingStudents performance and students learningȘtefan et al. [80]9QualitativeNot mentionedNot mentionedSeverengiz, Seliger, and Krüger [38]13Postgame surveys from 31 studentsDescriptive studyLoon, Evans, and Kerridge [81]14Mixed methodsPostgame surveys 155 and 36 semistructured interviews from studentsPearson correlation and multiple regression analysis SPSSBell and Loon [82]14QuantitativePostgame surveys from 173 studentsPrincipal component analysis, correlation and regression analysisEngagement, cognitive maturity, innovativeness, intended learning outcomesLee, Long, and Visinescu [83]13QuantitativePostgame surveys from 93 studentsPartial least square approachActive learning, meaningful learning, collaboration, subject integrationPando-Garcia, Periañez-Cañadillas, and Charterina [84]11QuantitativePostgame surveys from two groups, 131 online, 83 onsite studentsConfirmatory factor analysisPerceived usefulness, perceived ease of use (PEOU), attitude to the business game technology, intention to use a business game technologyLin, Yen, and Wang [85]10QuantitativeExperimental scenarios, an achievement test and a motivation scale, 49 response from individual and 47 response from groups of studentsDescriptive analysis, two-way ANOVA on SPSSLearning methods (collaborative and individual), learning motivation, learning performanceHwang and Cruthirds [86]12QuantitativePre-postgame surveys from 52 from studentsDescriptive analysisSAP ease of use, business process knowledge, enterprise system knowledge, sap transaction knowledgeUrquidi Martín, and Tamarit Aznar [87]10QuantitativePostgame survey from 58 studentsDescriptive analysisEvaluation of the experience, learning, and development of critical thinkingMustata et al. [88]10QualitativePostgame qualitative surveys from 88 studentsThematic analysisNewbery et al. [89]10QuantitativePre-postgame surveys from 236 students divided in treatment and control groupst-test and regressionGroup level micro identity, individual level micro identity, and interpersonal micro identity, observed entrepreneurial behaviour, experienced entrepreneurial behavioursTorres and Augusto [90]12QuantitativePostgame surveys from 22 MBA executive studentsANOVAStrategy formulation, strategy implementation, critical analysisFarashahi and Tajeddin [91]14QuantitativePostgame surveys from 194 undergraduate and MBA studentsDescriptive analysisProblem-solving skills, interpersonal skills, and self-awarenessAlas et al. [92]9Quantitative28 game dataOn SPSS, correlation analyses, regression analyses, andt-testsGPA, expenditure of market information, point for homework, normalized profitsCalabor, Mora, and Moya [93]14Quantitative12 academics DelphiDescriptive analysisTechnical aspects of game, learning values of games, general view of gameZulfiqar et al. [57]12QuantitativeTime-lagged surveys from 277 studentsStructural equation modellingKnowledge sharing, knowledge application, learnability, perceived pleasure, and self-efficacyBeranič and Heričko [62]12QuantitativePre-postgame survey from 32 students involved in ERPsim introductory simulationDescriptive analysisBusiness process knowledge, technical SAP knowledge, ERP transaction knowledge, intent for future course engagementSamaras, Adkins, and White [94]13QuantitativePostsurvey from 119 studentsPaired-samplet-testsCritical thinking process and simulation participation


## 4.3. Results of the Quality Assessment

The researchers selected the papers based on quality ratings. The minimum acceptable number was 9, and the maximum number given on the paper was 15. The average quality rating of the selected papers is 11. A standard deviation of 1.7 was found in the quality ratings for the total mean score. As shown in Figure10, there has been a steady increase in the quality of empirical papers on BSGs over time.Figure 10
Mean quality rating per year.


## 5. Discussion and Findings

In this literature review on the use of BSGs for teaching and learning, we found that the research on BSGs in teaching and learning has grown in the last five years. The quality rating of the selected papers is also increasing, as shown in Figure10. Most empirical studies explored the learning outcomes of incorporating these games in coursework. Using the key terms, the initial literature search retrieved 523 relevant papers, which were later reduced to the final sample of high-quality forty-nine open access empirical studies. The factor analysis of the selected studies provided a framework for organising the diverse research on BSGs and summarising trends in literature. This section presents the selected papers’ findings, limitations, and future recommendations of the selected papers. Four distinctive trends emerged from the literature review, which was used to analyse the selected studies:
(1)
Integration of BSGs in teaching and learning processes(2)
Learning outcomes achieved by using BSGs as a learning tool(3)
Learning theories used in BSG literature(4)
Evaluation methods used to assess the learning outcomes of BSGs and the game performance


### 5.1. Integration of BSGs in Teaching and Learning

The way simulation games are integrated into teaching and learning makes a difference in the success of achieving desired learning outcomes [95]. In a literature review, Oliveira et al. [96] addressed seven emerging technologies utilised in education, one of which is simulations. According to the research, simulations have been shown to improve students’ learning experiences, particularly in terms of problem-solving and creative thinking. An important enabling component for the successful use of simulation games in education is the role of the instructor/facilitator. Seven studies discussed the instructor’s part in achieving desired learning outcomes using BSGs.BSGs are often a supplementary tool to assist educators on regular courses. They are mainly conducted after lectures and tutorials, usually at the end of the semester as a final test or integrational activity. They are also taught as separate courses in a degree program to complement the other courses [97]. In most selected studies, off-the-shelf BSGs are incorporated as an add-on or run as a test activity. Research on BSGs has proven that these games can help instructors facilitate students’ engagement, learning, and employability if they are appropriately integrated [4]. Since a BSG consists of many decision-making elements (e.g., finance, marketing, human resources, distribution, and logistics), it is essential for instructors first to identify the learning outcomes they want to achieve and then use BSGs as a mean to achieve those outcomes [98].Vos [43] explained the three important roles instructors have to play in integrating and supporting these learning activities: instructors have to plan appropriate learning and teaching strategies, support students during the whole process, and then design appropriate assessment tasks to assess whether the game is achieving the required learning outcomes. Ellahi et al. [65] reinforced the instructors’ role in incorporating BSGs into courses. The study used instructor support as a moderating factor to examine the relationship between simulation games and learning outcomes. The study’s findings showed that although the role of the instructor is essential in the learning process using computer simulations, the research argued that students should mainly take control of their learning. On the contrary, Hernández-Lara and Serradell-López [41] suggested that instructors should play an active role at the pregame stage and then throughout the game while determining the complexity levels and other conditions of the game that include the economic environment, market conditions, production facilities, employee engagement and motivation, and other conditions. Loon et al. [81] conducted a mixed-methods study on the role of instructional design and the context of the study in strategic decision-making. He suggested that a lack of support during the simulation activity might result in demotivating the students from taking part in the activity. Even if they participate, the absence of instructor support can negate any possibility of learning from the process. He suggested three types of support for students during the activity: interpretative support (background information and relevant input knowledge, including elaborative and explanatory feedback), experimental support (in developing perspectives and propositions), and reflective support (inquiry process and knowledge gained from the simulation).Another important factor concerning the instructors’ role in the successful integration of BSGs was discussed by Kriz and Auchter [42] in an empirical study where they explored the effect of simulation games on entrepreneurial competencies by conducting more than 12000 surveys with students who have played different BSGs. One factor that influenced students’ acceptance of the simulation activity was the role of the instructors. According to the students, instructors’ skill levels, enthusiasm, and teaching quality impacted how students perceived and adopted these simulations. Pando-Garcia et al. [84] explored the effect of perceived usefulness and ease of use of simulation on the attitude and intention to use simulation by students using the Technology Acceptance Model (TAM) (Urquidi-Martín, Tamarit-Aznar, & Sánchez-García) model. TAM is a theory of information systems that describes how people learn to accept and use technology. The study’s results showed a strong relationship between positive attitudes towards using simulation games and their perceived ease of use. This leads the authors to conclude that the instructors can influence students’ attitudes toward using simulation because they can help students understand the game’s technical aspects, which may make the game experience easier for students. Overall, the selected studies which explored instructors’ role in using BSGs established the instructor’s significant role in the integration or running phases of the BSGs.Kriz and Auchter [42] explored the barriers to integrating BSGs in existing courses as part of their longitudinal study with multiple business schools all over Europe using BSGs. They identified the three most common barriers to adopting BSGs: the perceived risk of adopting new teaching techniques, the suitability of the available simulations, and lack of resources (financial and infrastructural). These three barriers are interlinked [99] and should be examined together. The risk perceptions of individual instructors vary and play an essential part in explaining the instructors’ attitude toward adopting the simulation. From an instructor’s perspective, as the simulation is a student-centred learning activity, there is the possibility that they may lose control over student learning.Rogmans and Abaza [69] conducted a comparative analysis of the effectiveness of two teaching methods in management education. The results of students’ surveys showed that the average engagement levels of students were higher in the traditional case study compared to simulation games. The findings showed that the main reason for students’ preference for selecting the case study over simulation is the complexity level of the simulation. The results showed that complexity was crucial in determining students’ acceptance of simulation-based learning. Calabor et al. [93] used Delphi techniques to explore instructors’ perceptions concerning barriers to incorporating BSGs in classrooms. In the panellists’ view, the most significant obstacle to implementing BSGs in existing courses was the lack of information on the suitability of the game for a particular course, the lack of infrastructure and financial resources, and the lack of knowledge regarding expected learning outcomes from the use of a specific BSG. They proposed to train instructors for new technologies and teaching methods so that they can facilitate their students’ learning performance. The researchers concluded that there is a general reluctance to adopt new teaching techniques and a need to create an awareness of the benefits for students and instructors of adopting simulation systems.The selected studies demonstrated the benefits of blended learning and incorporating BSGs into existing courses and traditional teaching methods (lectures and tutorials). For example, in one of the selected papers, Loon et al. [81] suggested the adoption of the guidelines provided by [100] and Salas et al. [101] in the selection and implementation of simulation games. The integrative aspect of blended learning was found to be effective as simulation games are not sufficient on their own to promote learning. They must be supported and supplemented by effective instructional designs and teaching approaches that involve regular intervention and instructor support through coaching and playing the role of a facilitator.


### 5.2. Learning Theories/Models and BSGs

Drawing on learning theory can assist educators in making well-informed decisions in selecting, integrating, and delivering specific simulation games. From the BSGs researchers’ point of view, learning theories were used to explain why certain aspects of simulations are successful or problematic and to develop a clear understanding of the research problems. Therefore, these researchers argued that BSGs provide a learning environment for students to bridge theoretical and conceptual knowledge taught in classrooms with authentic, real-world experiences [102].Several theories/models have been proposed in the selected studies for understanding the role of BSGs in enhancing learning and teaching processes. Fourteen studies used learning theory to develop a well-defined research model out of forty-nine chosen papers. Kolb’s [103] experiential learning theory is the most commonly used approach, followed by the Technology Acceptance Model [104]. The list of these fourteen papers, learning theories/models, and the constructs used is presented in Table 6.Table 6
Learning theories and the variable used in selected papers.
AuthorTheoriesConstructs used in studyIndependentDependentChen, Keys, and Gaber [44]Theory of planned behaviour [105]Behavioural intentions, enjoyment, cognitive appraisalPerceived learning outcomesBitrián, Buil, and Catalán [47]Flow theory [106]Boredom, flow, anxiety, apathyStudents’ perceived learning, skills, and satisfactionNewbery et al. [89]The models of entrepreneurial intent [107]Personal attitudes, perceived behavioural control, perceived social normsEntrepreneurial intentTao, Yeh, and Hung [64]Problem-based gaming model [108], input-process-outcome game model [109]Student-perceived complexity level of BSGs, BSG competition outcomesDeclarative knowledge, procedural knowledge, strategic knowledgeEllahi, Zaka, and Sultan [65]Activity theory [110]Learners’ interest, instructor roleSatisfaction and performanceWilliams [66]Experiential learning theory [103]Game cycleEntrepreneurial skills, entrepreneurial attitudesZulfiqar et al. [68]Technology Acceptance Model [104] and theory of planned behaviour [105]Perceived business value, subjective norms, perceived behavioural controls, attitudeEntrepreneurship and entrepreneurial intentionsBuil, Catalán, and Martínez [76]Self-system model of motivational development [111]Competence, autonomy, relatedness, self-efficacyCognitive, emotional, and behavioural engagement, skill development, perceived learningLeal-Rodriguez and Albort-Morant [79]Experiential learning theory [103]Students’ experiential learning involvement (project grade)Performance in the final exam (exams grade)Lee, Long, and Visinescu [83]Expectancy–value theory of motivation [112]Active learningMeaningful learningCollaboration, subject integrationBusiness intelligence motivational beliefsPando-Garcia, Periañez-Cañadillas, and Charterina [84]Technology Acceptance Model [104]Perceived usefulness, perceived ease of useAttitude to using BSG, intention to use a BSGNewbery et al. [89]Identity conflict theory [113]Group-level micro identity, individual micro identity, interpersonal micro identityObserved entrepreneurial identity, entrepreneurial behaviourTorres and Augusto [90]Experiential learning theory [103]Decision styles (analytical, conceptual, directive, behavioural)Strategic competencies (strategic formulation, strategic implementation, critical analysis)Urquidi-Martín, Tamarit-Aznar, and Sánchez-García [54]Experiential learning theory [103]Game realism, game structure, perceived usefulnessMotivation and critical thinkingWang et al. [70]Unified Theory of Acceptance and Use of Technology (UTAUT) [114]Performance and effort expectancy, social influence, facilitating conditions, hedonic motivation, price value and habitBehavioural intentionZulfiqar et al. [57]Technology Acceptance Model [104]Perceived usefulness, perceived ease of use, technology adoptionEntrepreneurial intention and learning performanceThe BSG literature uses the learning paradigms of humanism, constructivism, cognitivism, and behaviourism to varying degrees. The fundamental theories that play a significant role in understanding learning during simulations are game-based, problem-based, and experiential learning.Many studies combined multiple theories. For example, Tao et al. [64] conducted an experimental study where he combined the problem-based gaming model [108] and the input-process-outcome game model [109] to examine the perceived learning-cycle effects caused by playing multiple BSGs with different complexity levels to develop the research model of the study. He used the problem-based gaming model to create constructs related to complexity and the input-process-outcome game model to develop constructs of knowledge gain (declarative, procedural, and strategic). The study’s findings showed that a significant number of students perceived that the complexity level of the BSG affected their knowledge levels. Zulfiqar et al. [68] combined the theory of planned behaviour [105] and the Technology Acceptance Model [104] to explore how the use of simulation games influenced and developed entrepreneurial attitudes in students. The variables measuring perceived business value came from the TAM, and subjective norms, perceived behavioural controls, and attitudes were taken from the theory of planned behaviour. The results showed a significant increase in the perceived behavioural controls and attitude toward entrepreneurial activities after using BSGs. However, subjective norms did not have a substantial effect on entrepreneurial attitudes.


### 5.3. Learning Outcomes of Using BSGs

A significant part of empirical studies on BSGs focused on measuring learning outcomes. Identifying learning outcomes enables instructors and researchers to determine how much progress students have made in meeting learning outcomes while completing a course. These outcomes consist of students’ knowledge and skills to acquire for effective learning. Many studies discussed the methodologies and benefits of integrating simulation games in different business courses. For the effective integration of simulation games in education, the specificity of the selected game regarding the desired learning outcomes to be considered. We observed that most chosen papers explained clearly defined learning objectives using a specific simulation game. Faisal et al. [61], for example, conducted qualitative research to investigate teachers’ perceptions of utilising ERPsim games in a supply chain training. The ERPsim game is an innovative learning method which provides students a real-time, risk-free environment to improve their knowledge of business processes and ERP systems [115]. ERPsim game runs on SAP HANA, the latest version of SAP as it provides an in-memory database which facilitates rapid processing and analysis of big data in real time [13]. This in-memory database provides students instant results of the decisions they have made [116]. This business simulation game has been used in more than 250 universities worldwide and a few universities in Australia including University of Melbourne, RMIT University, Victoria University, and Federation University, to teach enterprise system concepts.Figure11 illustrates the scope of the manufacturing simulation game. Shaded rectangles represent SAP transactions automated by ERPsim, while students must perform other transactions to manage their company.Figure 11
ERPsim manufacturing game.These games were also designed to provide a rich and economically representative simulation of a medium-sized organisation. The most measured skills in the selected studies are communication, decision-making, entrepreneurial skills, analytical thinking, and problem-solving skills. The most common knowledge outcome is business process knowledge discussed in 9 studies, followed by ICT and functional knowledge (5 studies) and supply chain knowledge (3 studies). The most common behavioural outcomes are engagement (12 studies), followed by motivation (7 studies) and self-efficacy (4 studies). Table7 shows the commonly measured learning outcomes and the respective studies they are discussed in.Table 7
Learning outcomes in selected studies.
Learning outcomesPapers measuring these outcomesCommunication[19, 45, 47, 50, 52, 54, 88, 90, 91, 117]Problem-solving[4, 19, 43, 50, 52, 54, 73, 81, 90, 91]Decision-making[19, 39, 41, 46, 47, 50, 52, 54, 68, 71, 78, 80, 88, 117]Analytical thinking[19, 43, 47, 50, 66, 81, 86, 88, 90]Teamwork[4, 19, 42, 43, 46, 47, 50, 54, 73, 80, 88, 118]Business process and operations knowledge[41, 42, 46, 50, 52, 54, 62, 86]Functional knowledge[19, 41, 50, 52, 62, 64, 66, 68, 71, 78–80, 119]Motivation[4, 41, 42, 64–66]Engagement[43, 46, 62, 65, 67, 68, 76, 78, 82, 83]Self-efficacy[50, 61, 64, 68, 88, 91]From the selected studies, the learning outcomes can be classified into three main categories: knowledge acquisition, skill development, and behavioural outcomes.


#### 5.3.1. Knowledge Gains from Using BSGs

Many reviewed papers discussed the impact of simulation games on knowledge acquisition. Almost all the selected papers reported the beneficial implications of the use of simulation games on knowledge acquisition and conceptual understanding. Specific simulation games were used in different courses to improve the content knowledge of students. For example, the pre-postgame survey’ results of a study by Beranič and Heričko [62] showed a positive impact on students’ knowledge in the business process and ERP transactions and in the domain of technical knowledge for SAP ERP. Similarly, a quantitative study by Angolia and Pagliari [119] discussed an ERP simulation game to aid students’ conceptual learning of ERP systems in a supply chain course. ERPsim was also reported to increase students’ knowledge of business processes. The study revealed that the simulation enhanced the distribution supply chain management’s concepts, which increased the course content’s understanding. One of the selected studies [46] also revealed knowledge gains in business processes and basic business concepts using ERPsim. Other studies confirmed the positive effect of using ERPsim games on business processes and supply chain knowledge [83, 86]. The ENTRExplorer game had a positive effect on students’ accounting knowledge and business processes knowledge (Fernando [52]). Our review of selected papers revealed that simulations are directly linked to the course content, and their use allows applying a better understanding of theoretical concepts [41, 45, 81].


#### 5.3.2. Skill Outcomes of Using BSGs

Cognitive abilities enable students to perform complex tasks such as logical critical thinking, problem-solving, and strategic decision-making [117]. These abilities are also sought after by the employers [120]. Twenty-eight papers were identified that reported the positive impact of using simulation games on students’ cognitive skills. For example, Levant et al. [73] conducted a pre-postsurvey to determine the effect of a business simulation game on students’ cognitive and interactive skills. This case study suggested positive results in the skills related to information management, communication, time management, project management, work-load management, team work, and problem-solving, which are considered higher-order cognitive skills [51]. The studies reporting the positive effects of using BSGs on the development and enhancement of cognitive skills are as follows:
(i)
Decision-making [39, 41, 46, 52, 67, 68, 73, 89, 117, 121](ii)
Critical thinking and scientific reasoning [19, 38, 43, 66, 81, 90](iii)
Problem-solving skills [19, 38, 43, 52, 54, 67, 73, 81, 90]Simulation games are often perceived as powerful learning tools to enhance students’ interactive or social skills as they are often played in teams. Most of the papers reported the positive impact of simulation games on different interactive skills.(i)
Teamwork [4, 19, 38, 41, 42, 46, 50, 73, 99, 122](ii)
Collaboration [43, 47, 80](iii)
Social and emotional skills [123](iv)
Learner-learner interaction [41]; the paper examined the impact of a simulation game called ERPsim by analysing the messages posted by the students on an online discussion forum during the game. They observed students’ active participation and social interaction and reported an overall positive impact on interactive skills. Other studies reported that the interactive skills enhanced by playing BSGs are communication, persuasion, and conflict resolution


#### 5.3.3. Behavioural Outcomes of BSGs

Many papers highlighted the behavioural and affective outcomes of computer simulation games.(i)
Positive effects were reported concerning students’ engagement with the simulation activity [43, 46, 62, 65, 67, 68, 76, 83](ii)
Increased motivation toward playing simulation [4, 41, 42, 64, 65, 69, 84](iii)
Higher satisfaction levels with the course content [4, 42, 47, 65, 84](iv)
Increased self-efficacy [39, 50, 64, 66, 68, 76](v)
Increased self-awareness and confidence [38, 39, 68, 91]To conclude, most of the selected papers reveal positive effects regarding the use of computer simulation games on learning outcomes. Figure12 shows the most common learning outcomes of BSGs identified in the selected papers of this review.Figure 12
Learning outcomes of BSGs.


### 5.4. Evaluation Methods Used in BSG Literature

Analysis of the selected studies shows the absence of a uniform agreed evaluation framework to assess the impact of using simulation games in learning [124]. Instructors use different assessment methods to evaluate the simulation activity performance of students and other evaluation methods used by the researchers to assess the impact of simulation activity on the learning outcomes in the selected studies.
(i)
Instructors used game results (profit gains) to indicate better game performance [39, 42, 79, 80, 83](ii)
Reflection reports submitted by the students act as a formative assessment for the game activity [19, 66]. Reflection reports usually contain questions related to students’ opinions on the simulation activities, their perceived gains from participating in the activity, their intention to use it again, the decisions they made, and the mistakes they wanted to rectify in the next session(iii)
Exam grades are used as performance indicators as objective measures to assess students’ knowledge gains [43, 79]During the pregame stage, questionnaires are the most frequently used assessment tools. Researchers can use this tool to collect data from many respondents regarding their previous knowledge, demographic information, perceptions of innovative pedagogical tools, and prior learning styles. This information can help educators set goals and the pace and pattern of the game as they can indicate students’ current knowledge and skill levels. Pregame surveys are rarely stand-alone. They are always followed by postgame surveys, which help researchers compare and analyse students’ knowledge gains and achievement of the desired learning outcomes. Out of forty-nine selected studies, six [47, 65, 66, 73, 86, 89] used pregame surveys followed by postgame surveys.The in-game assessments include observations of players while playing the game, in-group discussion, and reflection reports. Classroom observation is helpful to obtain instant feedback on game experience and enable a cognitive analysis of students as players to verbalise their thinking and decision-making process during discussions. In addition, observations provide insight into students’ interaction with the interface and other game designs [85]. Reflection reports are also a part of in-game assessment where students qualitatively assess their learning outcomes, experience, and knowledge gain. Reflection reports can be conducted once or multiple times during the game. Overall, in-game assessments measure students’ mood, understanding, learning, and performance. The literature search revealed the importance of in-game assessment as only this can provide instant feedback about overall learning processes and other behavioural changes resulting from this process [125]. However, it is the least adopted method in BSG literature because of its complexity, time constraints, and issues regarding permission to access participants.Postgame assessments are the most used evaluation methods in the selected studies (45) and assess students’ actual knowledge gain and improvement in skills. In this assessment, games are evaluated considering their usefulness in achieving desired learning outcomes. Questionnaires are the most common tool for postgame assessment, followed by interviews and focus groups. Although most studies used quantitative surveys to conduct postgame assessments, a few used qualitative methods. For example, Loon et al. [81] conducted interviews with 36 students to explore their simulation activity views after they had taken part in the simulations. Goi [4] conducted focus group discussions with 17 students who played Total Enterprise Simulation as part of an undergraduate course to explore the role of authentic team-based learning in enhancing students’ learning outcomes and satisfaction. In several studies, game results (financial and nonfinancial indicators) are also used as postgame assessments [39, 66, 79, 92]).The most popular assessment tool is questionnaires due to their ease of use with larger sample sizes. However, qualitative assessments such as reflection reports, classroom observation, and interviews give more profound insight into the learning experience [126]. In the research, there was not an assessment framework used to evaluate BSGs, as most studies used self-developed frameworks. This lack of a univocal assessment framework raises questions about the validity of the results. The analysis of the 57 selected papers shows that the evaluations are done at different phases of BSGs. Researchers used other qualitative and quantitative measurements to assess the impact of BSGs on learning outcomes. Based on the analysis, researchers presented a comprehensive evaluation framework (Figure 13) to evaluate the learning outcomes of simulation games. Instead of focusing on just a single point of time. Different learning outcomes are gained at various game stages; this framework can help educators assess them all.Figure 13
Evaluation methods at different stages of BSGs.


## 5.1. Integration of BSGs in Teaching and Learning

The way simulation games are integrated into teaching and learning makes a difference in the success of achieving desired learning outcomes [95]. In a literature review, Oliveira et al. [96] addressed seven emerging technologies utilised in education, one of which is simulations. According to the research, simulations have been shown to improve students’ learning experiences, particularly in terms of problem-solving and creative thinking. An important enabling component for the successful use of simulation games in education is the role of the instructor/facilitator. Seven studies discussed the instructor’s part in achieving desired learning outcomes using BSGs.BSGs are often a supplementary tool to assist educators on regular courses. They are mainly conducted after lectures and tutorials, usually at the end of the semester as a final test or integrational activity. They are also taught as separate courses in a degree program to complement the other courses [97]. In most selected studies, off-the-shelf BSGs are incorporated as an add-on or run as a test activity. Research on BSGs has proven that these games can help instructors facilitate students’ engagement, learning, and employability if they are appropriately integrated [4]. Since a BSG consists of many decision-making elements (e.g., finance, marketing, human resources, distribution, and logistics), it is essential for instructors first to identify the learning outcomes they want to achieve and then use BSGs as a mean to achieve those outcomes [98].Vos [43] explained the three important roles instructors have to play in integrating and supporting these learning activities: instructors have to plan appropriate learning and teaching strategies, support students during the whole process, and then design appropriate assessment tasks to assess whether the game is achieving the required learning outcomes. Ellahi et al. [65] reinforced the instructors’ role in incorporating BSGs into courses. The study used instructor support as a moderating factor to examine the relationship between simulation games and learning outcomes. The study’s findings showed that although the role of the instructor is essential in the learning process using computer simulations, the research argued that students should mainly take control of their learning. On the contrary, Hernández-Lara and Serradell-López [41] suggested that instructors should play an active role at the pregame stage and then throughout the game while determining the complexity levels and other conditions of the game that include the economic environment, market conditions, production facilities, employee engagement and motivation, and other conditions. Loon et al. [81] conducted a mixed-methods study on the role of instructional design and the context of the study in strategic decision-making. He suggested that a lack of support during the simulation activity might result in demotivating the students from taking part in the activity. Even if they participate, the absence of instructor support can negate any possibility of learning from the process. He suggested three types of support for students during the activity: interpretative support (background information and relevant input knowledge, including elaborative and explanatory feedback), experimental support (in developing perspectives and propositions), and reflective support (inquiry process and knowledge gained from the simulation).Another important factor concerning the instructors’ role in the successful integration of BSGs was discussed by Kriz and Auchter [42] in an empirical study where they explored the effect of simulation games on entrepreneurial competencies by conducting more than 12000 surveys with students who have played different BSGs. One factor that influenced students’ acceptance of the simulation activity was the role of the instructors. According to the students, instructors’ skill levels, enthusiasm, and teaching quality impacted how students perceived and adopted these simulations. Pando-Garcia et al. [84] explored the effect of perceived usefulness and ease of use of simulation on the attitude and intention to use simulation by students using the Technology Acceptance Model (TAM) (Urquidi-Martín, Tamarit-Aznar, & Sánchez-García) model. TAM is a theory of information systems that describes how people learn to accept and use technology. The study’s results showed a strong relationship between positive attitudes towards using simulation games and their perceived ease of use. This leads the authors to conclude that the instructors can influence students’ attitudes toward using simulation because they can help students understand the game’s technical aspects, which may make the game experience easier for students. Overall, the selected studies which explored instructors’ role in using BSGs established the instructor’s significant role in the integration or running phases of the BSGs.Kriz and Auchter [42] explored the barriers to integrating BSGs in existing courses as part of their longitudinal study with multiple business schools all over Europe using BSGs. They identified the three most common barriers to adopting BSGs: the perceived risk of adopting new teaching techniques, the suitability of the available simulations, and lack of resources (financial and infrastructural). These three barriers are interlinked [99] and should be examined together. The risk perceptions of individual instructors vary and play an essential part in explaining the instructors’ attitude toward adopting the simulation. From an instructor’s perspective, as the simulation is a student-centred learning activity, there is the possibility that they may lose control over student learning.Rogmans and Abaza [69] conducted a comparative analysis of the effectiveness of two teaching methods in management education. The results of students’ surveys showed that the average engagement levels of students were higher in the traditional case study compared to simulation games. The findings showed that the main reason for students’ preference for selecting the case study over simulation is the complexity level of the simulation. The results showed that complexity was crucial in determining students’ acceptance of simulation-based learning. Calabor et al. [93] used Delphi techniques to explore instructors’ perceptions concerning barriers to incorporating BSGs in classrooms. In the panellists’ view, the most significant obstacle to implementing BSGs in existing courses was the lack of information on the suitability of the game for a particular course, the lack of infrastructure and financial resources, and the lack of knowledge regarding expected learning outcomes from the use of a specific BSG. They proposed to train instructors for new technologies and teaching methods so that they can facilitate their students’ learning performance. The researchers concluded that there is a general reluctance to adopt new teaching techniques and a need to create an awareness of the benefits for students and instructors of adopting simulation systems.The selected studies demonstrated the benefits of blended learning and incorporating BSGs into existing courses and traditional teaching methods (lectures and tutorials). For example, in one of the selected papers, Loon et al. [81] suggested the adoption of the guidelines provided by [100] and Salas et al. [101] in the selection and implementation of simulation games. The integrative aspect of blended learning was found to be effective as simulation games are not sufficient on their own to promote learning. They must be supported and supplemented by effective instructional designs and teaching approaches that involve regular intervention and instructor support through coaching and playing the role of a facilitator.


## 5.2. Learning Theories/Models and BSGs

Drawing on learning theory can assist educators in making well-informed decisions in selecting, integrating, and delivering specific simulation games. From the BSGs researchers’ point of view, learning theories were used to explain why certain aspects of simulations are successful or problematic and to develop a clear understanding of the research problems. Therefore, these researchers argued that BSGs provide a learning environment for students to bridge theoretical and conceptual knowledge taught in classrooms with authentic, real-world experiences [102].Several theories/models have been proposed in the selected studies for understanding the role of BSGs in enhancing learning and teaching processes. Fourteen studies used learning theory to develop a well-defined research model out of forty-nine chosen papers. Kolb’s [103] experiential learning theory is the most commonly used approach, followed by the Technology Acceptance Model [104]. The list of these fourteen papers, learning theories/models, and the constructs used is presented in Table 6.Table 6
Learning theories and the variable used in selected papers.
AuthorTheoriesConstructs used in studyIndependentDependentChen, Keys, and Gaber [44]Theory of planned behaviour [105]Behavioural intentions, enjoyment, cognitive appraisalPerceived learning outcomesBitrián, Buil, and Catalán [47]Flow theory [106]Boredom, flow, anxiety, apathyStudents’ perceived learning, skills, and satisfactionNewbery et al. [89]The models of entrepreneurial intent [107]Personal attitudes, perceived behavioural control, perceived social normsEntrepreneurial intentTao, Yeh, and Hung [64]Problem-based gaming model [108], input-process-outcome game model [109]Student-perceived complexity level of BSGs, BSG competition outcomesDeclarative knowledge, procedural knowledge, strategic knowledgeEllahi, Zaka, and Sultan [65]Activity theory [110]Learners’ interest, instructor roleSatisfaction and performanceWilliams [66]Experiential learning theory [103]Game cycleEntrepreneurial skills, entrepreneurial attitudesZulfiqar et al. [68]Technology Acceptance Model [104] and theory of planned behaviour [105]Perceived business value, subjective norms, perceived behavioural controls, attitudeEntrepreneurship and entrepreneurial intentionsBuil, Catalán, and Martínez [76]Self-system model of motivational development [111]Competence, autonomy, relatedness, self-efficacyCognitive, emotional, and behavioural engagement, skill development, perceived learningLeal-Rodriguez and Albort-Morant [79]Experiential learning theory [103]Students’ experiential learning involvement (project grade)Performance in the final exam (exams grade)Lee, Long, and Visinescu [83]Expectancy–value theory of motivation [112]Active learningMeaningful learningCollaboration, subject integrationBusiness intelligence motivational beliefsPando-Garcia, Periañez-Cañadillas, and Charterina [84]Technology Acceptance Model [104]Perceived usefulness, perceived ease of useAttitude to using BSG, intention to use a BSGNewbery et al. [89]Identity conflict theory [113]Group-level micro identity, individual micro identity, interpersonal micro identityObserved entrepreneurial identity, entrepreneurial behaviourTorres and Augusto [90]Experiential learning theory [103]Decision styles (analytical, conceptual, directive, behavioural)Strategic competencies (strategic formulation, strategic implementation, critical analysis)Urquidi-Martín, Tamarit-Aznar, and Sánchez-García [54]Experiential learning theory [103]Game realism, game structure, perceived usefulnessMotivation and critical thinkingWang et al. [70]Unified Theory of Acceptance and Use of Technology (UTAUT) [114]Performance and effort expectancy, social influence, facilitating conditions, hedonic motivation, price value and habitBehavioural intentionZulfiqar et al. [57]Technology Acceptance Model [104]Perceived usefulness, perceived ease of use, technology adoptionEntrepreneurial intention and learning performanceThe BSG literature uses the learning paradigms of humanism, constructivism, cognitivism, and behaviourism to varying degrees. The fundamental theories that play a significant role in understanding learning during simulations are game-based, problem-based, and experiential learning.Many studies combined multiple theories. For example, Tao et al. [64] conducted an experimental study where he combined the problem-based gaming model [108] and the input-process-outcome game model [109] to examine the perceived learning-cycle effects caused by playing multiple BSGs with different complexity levels to develop the research model of the study. He used the problem-based gaming model to create constructs related to complexity and the input-process-outcome game model to develop constructs of knowledge gain (declarative, procedural, and strategic). The study’s findings showed that a significant number of students perceived that the complexity level of the BSG affected their knowledge levels. Zulfiqar et al. [68] combined the theory of planned behaviour [105] and the Technology Acceptance Model [104] to explore how the use of simulation games influenced and developed entrepreneurial attitudes in students. The variables measuring perceived business value came from the TAM, and subjective norms, perceived behavioural controls, and attitudes were taken from the theory of planned behaviour. The results showed a significant increase in the perceived behavioural controls and attitude toward entrepreneurial activities after using BSGs. However, subjective norms did not have a substantial effect on entrepreneurial attitudes.


## 5.3. Learning Outcomes of Using BSGs

A significant part of empirical studies on BSGs focused on measuring learning outcomes. Identifying learning outcomes enables instructors and researchers to determine how much progress students have made in meeting learning outcomes while completing a course. These outcomes consist of students’ knowledge and skills to acquire for effective learning. Many studies discussed the methodologies and benefits of integrating simulation games in different business courses. For the effective integration of simulation games in education, the specificity of the selected game regarding the desired learning outcomes to be considered. We observed that most chosen papers explained clearly defined learning objectives using a specific simulation game. Faisal et al. [61], for example, conducted qualitative research to investigate teachers’ perceptions of utilising ERPsim games in a supply chain training. The ERPsim game is an innovative learning method which provides students a real-time, risk-free environment to improve their knowledge of business processes and ERP systems [115]. ERPsim game runs on SAP HANA, the latest version of SAP as it provides an in-memory database which facilitates rapid processing and analysis of big data in real time [13]. This in-memory database provides students instant results of the decisions they have made [116]. This business simulation game has been used in more than 250 universities worldwide and a few universities in Australia including University of Melbourne, RMIT University, Victoria University, and Federation University, to teach enterprise system concepts.Figure11 illustrates the scope of the manufacturing simulation game. Shaded rectangles represent SAP transactions automated by ERPsim, while students must perform other transactions to manage their company.Figure 11
ERPsim manufacturing game.These games were also designed to provide a rich and economically representative simulation of a medium-sized organisation. The most measured skills in the selected studies are communication, decision-making, entrepreneurial skills, analytical thinking, and problem-solving skills. The most common knowledge outcome is business process knowledge discussed in 9 studies, followed by ICT and functional knowledge (5 studies) and supply chain knowledge (3 studies). The most common behavioural outcomes are engagement (12 studies), followed by motivation (7 studies) and self-efficacy (4 studies). Table7 shows the commonly measured learning outcomes and the respective studies they are discussed in.Table 7
Learning outcomes in selected studies.
Learning outcomesPapers measuring these outcomesCommunication[19, 45, 47, 50, 52, 54, 88, 90, 91, 117]Problem-solving[4, 19, 43, 50, 52, 54, 73, 81, 90, 91]Decision-making[19, 39, 41, 46, 47, 50, 52, 54, 68, 71, 78, 80, 88, 117]Analytical thinking[19, 43, 47, 50, 66, 81, 86, 88, 90]Teamwork[4, 19, 42, 43, 46, 47, 50, 54, 73, 80, 88, 118]Business process and operations knowledge[41, 42, 46, 50, 52, 54, 62, 86]Functional knowledge[19, 41, 50, 52, 62, 64, 66, 68, 71, 78–80, 119]Motivation[4, 41, 42, 64–66]Engagement[43, 46, 62, 65, 67, 68, 76, 78, 82, 83]Self-efficacy[50, 61, 64, 68, 88, 91]From the selected studies, the learning outcomes can be classified into three main categories: knowledge acquisition, skill development, and behavioural outcomes.


### 5.3.1. Knowledge Gains from Using BSGs

Many reviewed papers discussed the impact of simulation games on knowledge acquisition. Almost all the selected papers reported the beneficial implications of the use of simulation games on knowledge acquisition and conceptual understanding. Specific simulation games were used in different courses to improve the content knowledge of students. For example, the pre-postgame survey’ results of a study by Beranič and Heričko [62] showed a positive impact on students’ knowledge in the business process and ERP transactions and in the domain of technical knowledge for SAP ERP. Similarly, a quantitative study by Angolia and Pagliari [119] discussed an ERP simulation game to aid students’ conceptual learning of ERP systems in a supply chain course. ERPsim was also reported to increase students’ knowledge of business processes. The study revealed that the simulation enhanced the distribution supply chain management’s concepts, which increased the course content’s understanding. One of the selected studies [46] also revealed knowledge gains in business processes and basic business concepts using ERPsim. Other studies confirmed the positive effect of using ERPsim games on business processes and supply chain knowledge [83, 86]. The ENTRExplorer game had a positive effect on students’ accounting knowledge and business processes knowledge (Fernando [52]). Our review of selected papers revealed that simulations are directly linked to the course content, and their use allows applying a better understanding of theoretical concepts [41, 45, 81].


### 5.3.2. Skill Outcomes of Using BSGs

Cognitive abilities enable students to perform complex tasks such as logical critical thinking, problem-solving, and strategic decision-making [117]. These abilities are also sought after by the employers [120]. Twenty-eight papers were identified that reported the positive impact of using simulation games on students’ cognitive skills. For example, Levant et al. [73] conducted a pre-postsurvey to determine the effect of a business simulation game on students’ cognitive and interactive skills. This case study suggested positive results in the skills related to information management, communication, time management, project management, work-load management, team work, and problem-solving, which are considered higher-order cognitive skills [51]. The studies reporting the positive effects of using BSGs on the development and enhancement of cognitive skills are as follows:
(i)
Decision-making [39, 41, 46, 52, 67, 68, 73, 89, 117, 121](ii)
Critical thinking and scientific reasoning [19, 38, 43, 66, 81, 90](iii)
Problem-solving skills [19, 38, 43, 52, 54, 67, 73, 81, 90]Simulation games are often perceived as powerful learning tools to enhance students’ interactive or social skills as they are often played in teams. Most of the papers reported the positive impact of simulation games on different interactive skills.(i)
Teamwork [4, 19, 38, 41, 42, 46, 50, 73, 99, 122](ii)
Collaboration [43, 47, 80](iii)
Social and emotional skills [123](iv)
Learner-learner interaction [41]; the paper examined the impact of a simulation game called ERPsim by analysing the messages posted by the students on an online discussion forum during the game. They observed students’ active participation and social interaction and reported an overall positive impact on interactive skills. Other studies reported that the interactive skills enhanced by playing BSGs are communication, persuasion, and conflict resolution


### 5.3.3. Behavioural Outcomes of BSGs

Many papers highlighted the behavioural and affective outcomes of computer simulation games.(i)
Positive effects were reported concerning students’ engagement with the simulation activity [43, 46, 62, 65, 67, 68, 76, 83](ii)
Increased motivation toward playing simulation [4, 41, 42, 64, 65, 69, 84](iii)
Higher satisfaction levels with the course content [4, 42, 47, 65, 84](iv)
Increased self-efficacy [39, 50, 64, 66, 68, 76](v)
Increased self-awareness and confidence [38, 39, 68, 91]To conclude, most of the selected papers reveal positive effects regarding the use of computer simulation games on learning outcomes. Figure12 shows the most common learning outcomes of BSGs identified in the selected papers of this review.Figure 12
Learning outcomes of BSGs.


## 5.3.1. Knowledge Gains from Using BSGs

Many reviewed papers discussed the impact of simulation games on knowledge acquisition. Almost all the selected papers reported the beneficial implications of the use of simulation games on knowledge acquisition and conceptual understanding. Specific simulation games were used in different courses to improve the content knowledge of students. For example, the pre-postgame survey’ results of a study by Beranič and Heričko [62] showed a positive impact on students’ knowledge in the business process and ERP transactions and in the domain of technical knowledge for SAP ERP. Similarly, a quantitative study by Angolia and Pagliari [119] discussed an ERP simulation game to aid students’ conceptual learning of ERP systems in a supply chain course. ERPsim was also reported to increase students’ knowledge of business processes. The study revealed that the simulation enhanced the distribution supply chain management’s concepts, which increased the course content’s understanding. One of the selected studies [46] also revealed knowledge gains in business processes and basic business concepts using ERPsim. Other studies confirmed the positive effect of using ERPsim games on business processes and supply chain knowledge [83, 86]. The ENTRExplorer game had a positive effect on students’ accounting knowledge and business processes knowledge (Fernando [52]). Our review of selected papers revealed that simulations are directly linked to the course content, and their use allows applying a better understanding of theoretical concepts [41, 45, 81].


## 5.3.2. Skill Outcomes of Using BSGs

Cognitive abilities enable students to perform complex tasks such as logical critical thinking, problem-solving, and strategic decision-making [117]. These abilities are also sought after by the employers [120]. Twenty-eight papers were identified that reported the positive impact of using simulation games on students’ cognitive skills. For example, Levant et al. [73] conducted a pre-postsurvey to determine the effect of a business simulation game on students’ cognitive and interactive skills. This case study suggested positive results in the skills related to information management, communication, time management, project management, work-load management, team work, and problem-solving, which are considered higher-order cognitive skills [51]. The studies reporting the positive effects of using BSGs on the development and enhancement of cognitive skills are as follows:
(i)
Decision-making [39, 41, 46, 52, 67, 68, 73, 89, 117, 121](ii)
Critical thinking and scientific reasoning [19, 38, 43, 66, 81, 90](iii)
Problem-solving skills [19, 38, 43, 52, 54, 67, 73, 81, 90]Simulation games are often perceived as powerful learning tools to enhance students’ interactive or social skills as they are often played in teams. Most of the papers reported the positive impact of simulation games on different interactive skills.(i)
Teamwork [4, 19, 38, 41, 42, 46, 50, 73, 99, 122](ii)
Collaboration [43, 47, 80](iii)
Social and emotional skills [123](iv)
Learner-learner interaction [41]; the paper examined the impact of a simulation game called ERPsim by analysing the messages posted by the students on an online discussion forum during the game. They observed students’ active participation and social interaction and reported an overall positive impact on interactive skills. Other studies reported that the interactive skills enhanced by playing BSGs are communication, persuasion, and conflict resolution


## 5.3.3. Behavioural Outcomes of BSGs

Many papers highlighted the behavioural and affective outcomes of computer simulation games.(i)
Positive effects were reported concerning students’ engagement with the simulation activity [43, 46, 62, 65, 67, 68, 76, 83](ii)
Increased motivation toward playing simulation [4, 41, 42, 64, 65, 69, 84](iii)
Higher satisfaction levels with the course content [4, 42, 47, 65, 84](iv)
Increased self-efficacy [39, 50, 64, 66, 68, 76](v)
Increased self-awareness and confidence [38, 39, 68, 91]To conclude, most of the selected papers reveal positive effects regarding the use of computer simulation games on learning outcomes. Figure12 shows the most common learning outcomes of BSGs identified in the selected papers of this review.Figure 12
Learning outcomes of BSGs.


## 5.4. Evaluation Methods Used in BSG Literature

Analysis of the selected studies shows the absence of a uniform agreed evaluation framework to assess the impact of using simulation games in learning [124]. Instructors use different assessment methods to evaluate the simulation activity performance of students and other evaluation methods used by the researchers to assess the impact of simulation activity on the learning outcomes in the selected studies.
(i)
Instructors used game results (profit gains) to indicate better game performance [39, 42, 79, 80, 83](ii)
Reflection reports submitted by the students act as a formative assessment for the game activity [19, 66]. Reflection reports usually contain questions related to students’ opinions on the simulation activities, their perceived gains from participating in the activity, their intention to use it again, the decisions they made, and the mistakes they wanted to rectify in the next session(iii)
Exam grades are used as performance indicators as objective measures to assess students’ knowledge gains [43, 79]During the pregame stage, questionnaires are the most frequently used assessment tools. Researchers can use this tool to collect data from many respondents regarding their previous knowledge, demographic information, perceptions of innovative pedagogical tools, and prior learning styles. This information can help educators set goals and the pace and pattern of the game as they can indicate students’ current knowledge and skill levels. Pregame surveys are rarely stand-alone. They are always followed by postgame surveys, which help researchers compare and analyse students’ knowledge gains and achievement of the desired learning outcomes. Out of forty-nine selected studies, six [47, 65, 66, 73, 86, 89] used pregame surveys followed by postgame surveys.The in-game assessments include observations of players while playing the game, in-group discussion, and reflection reports. Classroom observation is helpful to obtain instant feedback on game experience and enable a cognitive analysis of students as players to verbalise their thinking and decision-making process during discussions. In addition, observations provide insight into students’ interaction with the interface and other game designs [85]. Reflection reports are also a part of in-game assessment where students qualitatively assess their learning outcomes, experience, and knowledge gain. Reflection reports can be conducted once or multiple times during the game. Overall, in-game assessments measure students’ mood, understanding, learning, and performance. The literature search revealed the importance of in-game assessment as only this can provide instant feedback about overall learning processes and other behavioural changes resulting from this process [125]. However, it is the least adopted method in BSG literature because of its complexity, time constraints, and issues regarding permission to access participants.Postgame assessments are the most used evaluation methods in the selected studies (45) and assess students’ actual knowledge gain and improvement in skills. In this assessment, games are evaluated considering their usefulness in achieving desired learning outcomes. Questionnaires are the most common tool for postgame assessment, followed by interviews and focus groups. Although most studies used quantitative surveys to conduct postgame assessments, a few used qualitative methods. For example, Loon et al. [81] conducted interviews with 36 students to explore their simulation activity views after they had taken part in the simulations. Goi [4] conducted focus group discussions with 17 students who played Total Enterprise Simulation as part of an undergraduate course to explore the role of authentic team-based learning in enhancing students’ learning outcomes and satisfaction. In several studies, game results (financial and nonfinancial indicators) are also used as postgame assessments [39, 66, 79, 92]).The most popular assessment tool is questionnaires due to their ease of use with larger sample sizes. However, qualitative assessments such as reflection reports, classroom observation, and interviews give more profound insight into the learning experience [126]. In the research, there was not an assessment framework used to evaluate BSGs, as most studies used self-developed frameworks. This lack of a univocal assessment framework raises questions about the validity of the results. The analysis of the 57 selected papers shows that the evaluations are done at different phases of BSGs. Researchers used other qualitative and quantitative measurements to assess the impact of BSGs on learning outcomes. Based on the analysis, researchers presented a comprehensive evaluation framework (Figure 13) to evaluate the learning outcomes of simulation games. Instead of focusing on just a single point of time. Different learning outcomes are gained at various game stages; this framework can help educators assess them all.Figure 13
Evaluation methods at different stages of BSGs.


## 6. Limitations and Future Research Directions in Selected Studies

This literature review of the BSG literature found three significant limitations that point to future research recommendations.(i)
The most common limitations in the selected studies are the limited context of the study [54, 64, 79, 81] and the small convenience sample size which created issues in generalizing the results [19, 43, 64–66, 78, 85, 89]Future recommendations: different studies proposed different recommendations to overcome these limitations. Vos [43] suggested future studies with a broader cross section of BSG users from business and other fields. The diverse sample will include participants with different characteristics, which may increase the chances of generalizability. Tao et al. [64] advised the expansion of the sample sizes in future studies to cover more classes in different universities for more in-depth and extensive analyses. Loon et al. [81] recommended collecting data from various sources (e.g. students’ academic records) to establish the relationship between learning through simulation and academic performance. Urquidi-Martín et al. [54] conducted postgame surveys from 58 students from a single university in Romania. She suggested future research comparing the impact of using BSGs in different cultures and countries.
(ii)
Another limitation discussed in selected studies is either including limited variables or ignoring moderating and meditating factors that can influence the effectiveness of using BSGs to enhance learning outcomesFuture recommendations: Chen et al. [44] only examined two factors (enjoyment and cognitive appraisal). He recommended that more elements be included in the learning outcomes model to understand learning behaviour and outcomes, for example, students’ concentration, curiosity, innovative attitudes, personal skills in IT, and understanding of business processes. [67] compared the learning outcomes of three teaching methods (lectures, case studies, and simulation) by conducting surveys with students and instructors. The study did not consider the wide range of individual and group-level variables affecting BSGs’ performance and students’ learning outcomes. To overcome this limitation, he suggested investigating the factors that help predict or influence the use of BSGs may impact students’ attainment of learning outcomes. Additional variables that can be analysed in the future were presented by Buil et al. [76]. The study focused on personal factors but not on contextual factors. For future studies, the study suggested that contextual variables should be analysed. Other learning components, such as retention or transfer learning, can be assessed objectively (e.g., students’ academic grades and application tests).
(iii)
Students’ perceptions and reports about learning outcomes were also a limitation in a few studiesFuture recommendations: Burdon and Munro [45] suggested further research to focus on students’ objective performance instead of just including students’ perceptions. Similarly, [47] only used students’ self-reported learning outcomes to measure simulation activity success. This can present a bias in the results as many factors can influence students’ responses. To reduce this bias, in future studies, researchers suggested using objective measures such as student grades and application tests to measure learning. Another study [65] advised conducting interviews with instructors in addition to surveys with students to determine teachers’ viewpoints concerning the use of digital games to facilitate learning. The researchers also suggested investigating factors such as students’ self-efficacy, population demographic characteristics, curriculum, and instructors’ experience with BSGs in future studies focusing on the impact of game play on learning efficiency. The results in another study [4] were only based on students’ self-reported measures obtained by conducting postgame surveys with students. However, other factors like differences in student demographic characteristics and teaching approaches used by instructors, which may impact learning outcomes, were not considered. To overcome these limitations, researchers suggested a future study where objective measures of learning gains, for example, grades, alongside self-reported measures, for instance, postgame surveys, can be used to reduce biases in studies. Another suggestion was to explore instructors’ views regarding teamwork facilitating the acquisition and development of learning outcomes when using online BSGs.The selected studies in this review presented myriad future research recommendations in the growing field of BSG research. These recommendations can further expand researchers’ and instructors’ knowledge to take advantage of the full potential of using BSGs in teaching and learning.


## 7. Conclusion

This paper has presented a systematic literature review of the empirical research on BSGs published between January 2015 and February 2022. The analysis of the fifty-seven selected studies, fulfilling the inclusion and exclusion criteria set by researchers, presented four significant themes within BSG research. These themes are techniques for the integration of BSGs in the existing curriculum, the use of BSGs to achieve learning outcomes, evaluation methods used in BSGs by instructors and researchers, and learning theories used to explain learning processes in BSG use. A significant part of BSG research is concerned with the use of BSGs to attain learning outcomes, the assessment of those outcomes, and integration of BSGs to achieve those leering outcomes. Figure14 presents an overview of the findings of this review. As presented, BSGs follow the experiential learning cycle and by doing so can be used to attain verity of learning outcomes at different stages of games. Educators usually use one or mix of assessment methods to evaluate the learning outcomes.Figure 14
Overview of review findings.All empirical studies included in this review focused on the impact that the use of BSGs might have on the achievement of different learning outcomes: skill development, knowledge acquisition, and/or behavioural outcomes. Three conclusions can be drawn from this review:(i)
The use of BSGs in higher education courses positively affects students’ achievement of learning outcomes(ii)
Three critical factors should be kept in mind while integrating BSGs in existing courses (integration process, the specificity of the game, and the instructors’ role)(iii)
There is a need for a univocal multistage evaluation framework (from both students’ and instructors’ perspectives) to assess how using BSGs might enhance learning outcomesLike all SLRs, this research has a few limitations that should be considered when evaluating the reported findings: limited search terms, period of the paper published, and the databases included. However, the articles in this SLR provide an overview of the recent empirical research on BSGs. Given the growing number of studies in this field, several papers that discuss the impact of computer simulation games on learning outcomes may have been omitted in this study. However, it would not have been a deliberate, systematic omission. The review excluded theoretical and opinion papers and only included empirical studies as it was necessary to ground our understanding of the effects of the use of BSGs on students’ attainment of learning outcomes in research evidence rather than suppositions.Another limitation is that the review only included open access papers. The choice regarding the inclusion of open access papers is motivated by their worldwide visibility without barriers. These open access publications reach nonscholarly, industry, and academic audiences in less developed areas of the world, where there is minimal access to commercial databases and journals [127]. The sample excluded dissertations, books, and grey literature. The significance of this is that the quality standards of peer-reviewed publications do not have to be fulfilled by grey literature. Articles reviewed by academics must conform to the standards of the peer reviewer and the editorial board responsible for ensuring the journal’s content and quality [128]. Therefore, many studies were excluded from the review, which might otherwise have provided additional insight to this review. It is important to note that, although not considered a literary form of publication, grey literature is often produced by experts in the field [129]. In future studies, the grey literature can be used to minimise publication bias, improve the comprehensiveness of the review, and offer a more informed image of the available evidence [130].Our review will contribute to the BSG literature by consolidating the latest evidence to help researchers to explore the effective use of BSGs to facilitate learning processes. The study has many methodological and applied implications. From a methodological standpoint, we suggest that researchers consider the concordance between learning outcomes and the evaluation measures used to assess them. While most of the papers used surveys to determine knowledge and skill gains, combining different learning measures could be more appropriate. We suggest a multistage multimethod evaluation strategy to capture the full potential of using the BSGs to enhance student attainment of learning outcomes as each stage presents an opportunity for students to gain different skills and knowledge. An authentic and embedded assessment should include multiple measurement procedures to cover various aspects of games [125], reduce bias, and increase the results’ validity. Another piece of advice for the researchers is to consider both instructors’ and students’ perspectives while evaluating the impact of using BSGs on student achievement of learning outcomes as it would reduce biases in results. From an applied standpoint, this review may help instructors choose the best possible BSG according to their needs. It may also allow universities to integrate these games effectively into existing graduate and undergraduate courses. Our findings present an extensive avenue for future research in the BSG field, especially regarding the effects of BSGs on employability and transferrable skills of graduates.



---

*Source: 1578791-2022-11-22.xml*