continue...

“”

The effectiveness of applied
learning: an empirical evaluation
using role playing in the classroom
Harneel Acharya, Rakesh Reddy and Ahmed Hussein
School of Engineering and Computing, National University,
San Diego, California, USA
Jaspreet Bagga
CGI Group, San Diego, California, USA, and
Timothy Pettit
Department of Management and Marketing, National University,
San Diego, California, USA
Abstract
Purpose – The purpose of this paper is to evaluate the effectiveness of role playing as an applied learning
technique for enhanced classroom experiences as compared to traditional lecture methods.
Design/methodology/approach – This study uses the pre-test/post-test design to conduct experiments
with several control and experimental groups. Subjects are graduate students in an MBA program at a
private, non-profit university in a traditional classroom setting.
Findings – Students in the experimental group gained significantly more knowledge (post-test minus pre-test
scores) – 45 percent higher – through participation in the role playing exercise as compared to the control group.
Research limitations/implications – This study represents only a single educational discipline
explored using a single role playing learning activity. Impacts on the long-term retention of the knowledge
should be studied further.
Practical implications – Educators should enhance their classroom experience with more applied learning
activities such as role playing in order to increase knowledge gain and potentially longer knowledge retention.
Originality/value – This study uses a customized role playing activity within a business curriculum as one
of many applied learning techniques. The value to students was shown by significantly higher gain in
knowledge while simultaneously enhancing their enjoyment of the classroom experience to potentially
encourage further lifelong learning.
Keywords Experimental design, Applied learning, Teaching styles, Enhanced learning, Role playing
Paper type Research paper
1. Introduction
In the era of globalization, the functions of business are dynamic, and their success depends
on how efficiently companies manage their inter-firm relationships with different channel
members within the supply chain. Be it a manufacturing or a service company, supply chain
management (SCM) plays a vital role in determining the performance of the business. The
current research is focused on preparing supply chain leaders to best establish and maintain
proper business relationships through the use of a selected applied learning technique – role
playing. This work will compare evolving thought on applied learning vs traditional lectures
and recommend an alternative way to prepare future managers to serve their supply chain.
Journal of Research in Innovative
Teaching & Learning
Vol. 12 No. 3, 2018
pp. 295-310
Emerald Publishing Limited
2397-7604
DOI 10.1108/JRIT-06-2018-0013
Received 26 June 2018
Revised 5 October 2018
Accepted 24 November 2018
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/2397-7604.htm
© Harneel Acharya, Rakesh Reddy, Ahmed Hussein, Jaspreet Bagga and Timothy Pettit. Published in
Journal of Research in Innovative Teaching & Learning. Published by Emerald Publishing Limited.
This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may
reproduce, distribute, translate and create derivative works of this article (for both commercial and
non-commercial purposes), subject to full attribution to the original publication and authors. The full
terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
295
The
effectiveness of
applied
learning
1.1 Background
With the rise in competition in the global market, SCM activities are becoming more
dynamic, complex and diversified. Therefore, it has become imperative to improve and
enhance the skills of managers to deal with the rise in challenges in the supply chain
(Kovacs and Pato, 2014; Kotzab et al., 2018). Further, the pace of change, explosion of
knowledge and technological revolution have had a great influence on the field of education,
raising awareness of the need for new pedagogical and learning approaches (Bates, 2005;
Zhang and Nunamaker, 2003). In order to prepare students as ready professionals, it
is vital that instructors of business courses like SCM know how to best prepare their
customers – the students.
Motivating students so that they can understand the importance of integrated-topic
scenarios is another issue of concern (Chang et al., 2009). Therefore, the courses taught at
educational institutions are changing their teaching patterns and utilize comprehensive
learning solutions instead of focusing on subject knowledge only (Ten Eyck, 2011;
Preziosi and Alexakis, 2011; Tran and Lewis, 2012; Westrup and Planander, 2013; Tabak
and Lebron, 2017; Paul and Ponnam, 2018). For instance, in order to determine the
quantities to produce, managers need to consider their decisions with regard to demand
forecasting, allocation of resources and inventory policy as well as scheduling of
manpower and machinery. Hence, it is important that future managers first study
different aspects of a decision, then consider them in totality and not in isolation. Several
opportunities do exist for advanced learning in applied settings, such as internships,
residencies and practical learning courses such as capstone projects with industry clients
(Brown et al., 2018; Silva et al., 2018; Bhattacharya and Neelam, 2018). However, this
option of applied learning is also available for basic and intermediate courses to meet the
changing demands of the student.
A vast body of literature evaluates the effectiveness of different practical and applied
approaches to learning that can be embraced by teaching institutes so that they can
contribute significantly in producing ready managers. The research by Alessi and
Trollip (1985), Martocchio and Webster (1992), Quinn (1996) and Li et al. (2017) shows the
use of applied gaming tools in teaching different subjects resulted in better knowledge
retention and have acted as useful pedagogical tools supplementing the traditional
classroom course material.
In light of the changing business environment and the complex dynamics of SCM, there
is a need to alter the current traditional pedagogy and equip educators with modern,
comprehensive, applied, scenario-based teaching techniques. Therefore, this project will test
the effectiveness and benefits of the technique of role playing as just one technique that can
be used to implement applied learning in the field of SCM.
1.2 Problem statement
The development of hands-on skills is challenging for many workers due to lack of
coaching, mentoring and applied learning opportunities. In order to improve productivity
and implement best practices, businesses require better results than standard classroom
lectures. This study consolidates a thorough review of the benefits of applied learning and
recommends ways in which traditional classroom sessions can be enhanced by using
applied learning solutions:
H1. An applied learning role playing workshop to teach a SCM concept will yield higher
levels of knowledge gain as compared to the same aspect of SCM taught through
traditional learning systems.
In the following sections, this paper will present a comprehensive review of literature regarding
applied learning, a research methodology, the analysis of the data and final conclusions.
296
JRIT&L
12,3
2. Literature review
The rise in globalization and need for innovation have marked significant changes in
supply chains. The increasing role of SCM is associated with the rise in integration among
different members in the supply chain (Christopher et al., 2011). In the case of a
manufacturing chain, the manufacturer does not reach directly to the consumers in order
to sell its products, and therefore needs channel partners such as wholesalers, distributors
and retailers to reach consumers. Being an intermediary, the retailer transfers information
about customer tastes and preferences and feedback about the products to the
manufacturer. Hence, resellers become a vital part of a SCM. Therefore, the success of the
manufacturer is highly dependent upon the relationship between the manufacturer their
channel partners in SCM. This project will explore best practices in educating future
supply chain managers on the specific process of establishing and maintaining successful
business relationships.
2.1 Effective learning and pedagogy
Effective learning and education is characterized by rigorous coursework and effective
pedagogical approaches that aim at enhancing the success level for all students (Rhodes,
2007). Hmelo-Silver and Barrows (2006) assume that teaching is a complex task and the
instructor should make use of a repertoire of strategies in order to make the learning
effective. Effective education is not just about teaching but learning. To enhance learning,
students should be equipped with the optimal level of information and assignments
(Preziosi and Alexakis, 2011). This means that the information imparted should be
justified and not overly demanding. In other words, if it is too demanding, the students
will be discouraged and if it is non-challenging, the interest level of the learners will tend
to be shattered.
Instructors should focus on enhanced learning that is possible by being flexible
and assimilating new subject content while adopting a variety of pedagogical
methodologies (Douglas et al., 2008). It is also vital that the teaching style of the
instructor should be effective enough that they can be well understood, and their lessons
can be applied by the learners (Gooden et al., 2009). As provided by Niemi et al. (2010),
effective education can be developed only when the students are involved in the
decision-making process of learning and their experiences and stories as learners are
heard. Ferguson et al. (2011) also supported this argument and focused on involving the
students’ opinions in generating meaningful dialogue and informed decision making.
Instructors should be flexible to adopt changes in their teaching practices after
understanding the experiences of students in order to develop an effective learning forum
(Robinson and Taylor, 2007).
Another tool toward developing effective learning sessions and pedagogy is the ability to
scaffold the learning of the students (Husbands and Pearce, 2012). The scaffolding for
learning can be provided by offering intellectual, social and emotional support ( James and
Pollard, 2011). This move will motivate the students to develop interest in their studies. Kim
and Hannafin (2011) have posited that the students who do not develop an interest in the
studies and learning process are not able to progress. Real-world scenarios are forums for
creating interest, motivation and scaffolding. These techniques culminate into a new
teaching method called applied learning.
2.2 Applied learning
Learning is known to be effective when the learner is not a passive recipient of knowledge
but is a proactive personality in the entire learning process. Active learning is
possible when the learner perceives the existence of a relationship between prior
knowledge and new knowledge (Christmas, 2014). Bringing the real world to the
297
The
effectiveness of
applied
learning
classroom is viewed as one of the crucial ways to promote effective learning. Therefore,
there is an absolute need to transcend from the traditional approaches and pedagogy to
new techniques such that the learners can use their learning to solve existential problems
(Hui and Koplin, 2011).
Learning-by-doing is considered an apt approach in today’s technology-driven world,
wherein continuous learning should be the premise for overall development of the
students (Serbessa, 2006). The pedagogical activity in the case of applied learning
should be active and involve concrete processes by focusing on the motto of learning by
doing (Schwartzman and Henry, 2009). The focus of effective education should be
centered toward the learners wherein the role of the instructor is to guide and facilitate the
learner and the learning process and not assert control or discharge lecturers toward
the targeted learning goals (Silcock and Brundrett, 2001). With the rise in globalization
and changing demands of the business world, the learning approaches of the students
may need to be altered to provide the best possible fit. In the light of this dynamism, the
student-centered approach has become a necessity while deciding upon different
learning styles to apply in the classroom (Hudson, 2009). Moreover, the minds of the
learners are complex and have a heterogeneous nature and therefore, an effective
classroom should consider the constructivist perspective of learning. In order to
develop a constructivist perspective of learning, it is vital to implement a student-centered
model of instruction which should blend differentiated curriculum and assessment
paradigms (Klein, 2003). The learning theory of constructivism aims at making
learning meaningful. As per this perspective, learning is a self-directed sociocultural
process in which the instructor assumes the role of a facilitator (Tobin and Tippins, 1993).
2.3 Tools of applied learning in SCM
SCM education techniques are similar to those of all other fields. As suggested by Phillips
(2005), applied learning can be achieved through chats, role plays and case studies for
application-based education (Miller et al., 2016). Further, additional tools of applied
learning adopted by Ruey (2010) include submitted artifacts, interviews, surveys and
observations. Simulation is another technique used for teaching useful learning concepts
using applied learning (Ten Eyck, 2011). Instructors can adopt different gaming tools to
facilitate real-time applied learning in teaching SCM, logistics management and business
relationship management (Lewis and Maylor, 2007; Quinn, 1996). These gaming tools are
effective in motivating active participation in the course as they foster competition and
add excitement in the learning process (Elgood, 1997). Further, Dempsey et al. (1996)
established that games lead to a playful learning environment that is goal-oriented,
challenging and promoting competitiveness.
Using simulation tools and models in teaching SCM is more insightful and
helps to evaluate alternatives and outcomes of supply chains which further enhances
performance (Persson and Araldi, 2009). Moreover, it becomes possible to use quantifying
measures for problem solving, which is a more reliable and objective approach in
teaching (Katsaliakia et al., 2014). Simulation-based models should be used with caution,
however, as the learners may find it difficult to understand the use and importance of
these pedagogies (Chwif et al., 2000). Therefore, it is imperative that the learners become
familiar with the use of these applied learning tools and have the competence for
successfully implementing these models (Katsaliakia et al., 2014). Hence, the role of
the instructor becomes important in explaining the need and the correct way to
implement these applied learning tools so that the learning benefits can be maximized.
The most popular simulation game used while teaching SCM is the MIT Beer Game, in
which production and distribution in a multi-stage distribution channel is facilitated
(Chang et al., 2009).
298
JRIT&L
12,3
2.4 Use of role playing as an effective tool for applied learning
Role playing as an active learning technique involves a high level of participation from
students (Howell, 1991; Westrup and Planander, 2013; Barnabe, 2016; Tabak and Lebron,
2017). Role playing is a group activity involving more than one person who assume different
roles in a given situation (Rao and Stupans, 2012) with the aim to acquire learning
experiences (Sogunro, 2004). As per Yardley-Matwiejczuk (1997), role playing puts the
people in “as if” situations through simulations and actions depending on specific events
and circumstances such that different behaviors, roles and arguments directly influence and
deepen the learning experience. The preparatory part of role playing is crucial as it involves
establishing the descriptions of roles along with the pre-requisites of the involved
participants (Westrup and Planander, 2013).
Learning in role playing is facilitated by observing as well as acting out the series of
events happening in the respective situation. Role playing helps students to understand
the dilemmas in situations and highlight the values of interpretation, which are not
possible to study in the traditional lecture mode of teaching (Bryant and Darwin, 2004).
As per Rao and Stupans (2012), role playing helps to provide a better understanding of
situations and augments that are critical to the subject matter. Yang and Quadir (2018)
found significant evidence to support language learning during the role playing game.
Further, McCarthy and Anderson (2000) undertook a study to measure the effectiveness of
role playing as compared with the traditional methods of teaching and concluded that the
active learning tool of role playing leads to better absorption and retention of information.
Not only is it more effective, role play provides a variation from the usual lecture-based
pedagogy and thus provides an applied and practical aspect to the theory taught in the
classroom (Sogunro, 2004). Role playing is also recognized as a pedagogical tool that helps
to depict the manner in which one acts, reacts and communicates in a social interaction
(Daly et al., 2009). Thereby along with critical thinking, it facilitates enhancement of
communication and inter-personal skills. Further, as per Rao and Stupans (2012), role
playing provides deeper insight about the business world and the students who take part
in it can learn about the situations, issues, alternatives and methods to tackle later in their
working life. This means of pedagogy hones the skill sets of students that are required in
future employment (Ruhanen, 2005). Sogunro (2004) posited that role playing helps
students view a specific situation from different perspectives and not just the single
perspective of the instructor or any individual. Therefore, it can be said that role playing
acts as an important active learning tool. However, there are drawbacks in regard to
student learning as related to the different knowledge background of each student
(Qiu et al., 2018). Another study explored role playing for learning in the marketing field to
help students create favorable customer experience without direct supervision from the
instructor (Paul and Ponnam, 2018). Business ethics is another well-suited educational
realm for role playing learning (Sauser and Sims, 2018). Role playing also helps students
to understand the feelings, values, attitudes and body language that provide the context of
the business situation at hand. This is very valuable in business education as role playing
supports cross-sectoral integration and help students experience the complexity in
decision-making (Ferrero et al., 2018). Moreover, as per Alkin and Christie (2002), role
playing is a useful tool in teaching conflict resolution and preparing students with the
necessary skills required for effective conflict management.
2.5 Knowledge retention
According to the Accreditation Council for Pharmacy Education (nd), schools and colleges
should make use of active learning methods so that critical thinking, problem-solving skills
and better retention of knowledge can take place. There have been many researchers who
concluded that there is a positive relationship between the collaborative and applied
299
The
effectiveness of
applied
learning
learning approaches and higher cognitive and affective outcomes (Tran and Lewis, 2012;
Chang et al., 2009; Li et al., 2017; Ferrero et al., 2018). The research has been supported
by Johnson and Johnson (2009) who posit that applied learning tools augment the
achievement rates of the students and increase their knowledge retention. It has also been
shown that cooperative simulations like role playing lead to better academic, social
and psychological advantages ( Johnson and Johnson, 2005). As per Beck and Chizhik (2008)
and Zain et al. (2009), academic performance improves drastically when the pedagogy
contains applied learning tools.
The pre-test and post-test experimental method is the best method to evaluate the
performance of students to adjust for previous knowledge of the learners while directly
measuring the gain of knowledge for each student (Kilic, 2008; Doymus et al., 2010). The
growing literature continues to support applied learning techniques, and this study explores
one such option in the realm of SCM education.
3. Methodology
The experiment for this project was conducted with 91 graduate students at a private,
non-profit university over the period of 1.5 years with seven different sections of the same
course, each course lasting four weeks. These seven classes were pre-selected to
approximately divide the sample into control and experimental groups. The control group
was comprised of 59 students who were first given a pre-test on the subject matter, followed
by the traditional mode of face-to-face lecture by the instructor using a PowerPoint
presentation, and finally each student was given a post-test to capture their knowledge
gained. The experimental group had 32 students, who were also given a pre-test and then
the learning was facilitated using the applied learning technique of role playing, and again a
post–test was conducted to assess the knowledge gain. The Appendix shows the questions
used for the evaluations, with post-test questions re-sorted.
3.1 Data collection
Traditional teaching for the Control Group used a 30-min lecture on the subject of
business relationships in the context of SCM, focusing on the partnership model (see
Lambert et al., 2004, 2010). In contrast, the applied learning was facilitated using a role
playing exercise for the experimental group. The exercise required a real-world business
relationship consisting of a seller and a buyer, as selected by the students. For example,
one student group self-selected the Coca-Cola Company selling soft drinks to the
McDonald’s Corporation – guidelines were that both companies be familiar to all students
in the class. The role playing exercise took 45 min, in which the first 10 min explained the
research study and the partnership model. Thereafter, the experimental classes were
randomly divided into two groups, whereby the first group played various roles in the
Coca-Cola Company (e.g. the supplier), such as the President/Chief Executive Officer,
Vice President for Supply Management, Vice President for Operations and Vice President
for Distribution, while the other group performed similar roles for the McDonald’s
Corporation (e.g. the buyer). Both the groups were then given time to find background
data on the company, such as identifying the actual president or CEO, location of the
headquarters, number of employees, number of stores, revenues and the mission
statement of the company. After this, based on the partnership model, the major drivers –
“compelling reasons for partnering with the other firm” (Lambert et al., 2004, p. 23) – were
identified and scored by each team. Once the drivers were identified by the teams
separately, a facilitator session was conducted, whereby both the teams were asked to rate
the facilitators jointly. They then selected the proper level of business relationship
based on the partnership model and discussed the appropriate actions that these
companies should undertake toward increasing performance otherwise not obtainable
300
JRIT&L
12,3
without a partnership. With such a process, students played an active role in gathering the
data and reaching their own conclusions based on their active participation.
3.2 Hypotheses testing
In order to meet the research objectives, the research hypotheses were framed as:
H2(a/b). There is a gain in knowledge during the learning activities (lecture and
workshop).
H3. There is a greater gain in knowledge for the workshop as compared to the lecture
learning activities.
Figure 1 shows the knowledge connections before and after the learning experience
for both groups. It should be noted that scoring was evaluated on a scale of 0 to 6
(see Appendix for sample instrument) with partial credit given as appropriate in
increments of 1/2, 1/3 and 1/6 points based on the question, so continuous variables were
assumed. However, results were also confirmed with the non-parametric Mann-Whitney
U-tests used when the data are ordinal or when the assumptions of the t-test are not met
(Boslaugh and Watters, 2008).
4. Analysis
This section analyzes the knowledge acquired during the learning sessions, followed by a
comparison of knowledge gain between the control and experimental groups. After all
experiments were conducted, data were reviewed to test assumptions of normalcy and
continuality. Figures 2 and 3 show slight skewness on the lower end for the control group
and slight skewness on the upper end for the experimental group, with no outliers except for
a single perfect score in the experimental group (role playing).
For a statistical test of the normality assumption, the Shapiro-Wilk Normality
Test (Shapiro and Wilk, 1965) was performed, with all four data sets as acceptable at the
po0.05 level (see Table I), which along with group sample size over 30, the normality
assumption holds.
It was next appropriate to verify the equivalency of the groups before making any
comparisons. This evaluation of the experimental group as compared to the control group
must be done prior to the implementation of the treatment. Therefore, the pre-test scores
were used as a direct measure of the level of knowledge of subjects prior to the
experimental treatment (e.g. students may have had previous experience with the material
or may have reviewed course material prior to the class). The below student’s t-test for
Control Group with Traditional lecture
Experimental Group with Role Playing Workshop
Learning
Learning
H2b
H2a
H3
+
+
+
Knowledge
Before
Knowledge
After
Knowledge
Before
Knowledge
After
Comparison of Learning
Figure 1.
Visualization
of hypotheses
301
The
effectiveness of
applied
learning
two-samples assuming unequal variances verifies this assumption in that the there is no
statistically significant difference in means between the two group’s pre-test average
scores (see Table II).
4.1 Evaluation of learning
The next step was to compute the gain in knowledge from both the traditional and applied
learning techniques, using each students’ pre- and post-test scores. To verify that learning
occurred in each instructional format, scores were tested again using a student’s t-test but
this time for paired samples as each subject’s pre- and post-test scores are not independent.
The results obtained for the lecture and workshop are given in Tables III and IV; using each
group’s average pre-test score as baseline, the control group scored 53 percent higher
after the traditional lecture while the experimental group scored 76 percent higher
after their role playing activity. A graphical interpretation can be seen in Figures 4 and 5.
6.0
5.0
4.0
3.0
2.0
1.0
0.0
Scores (0–6 scale)
Pre-test Post-test
Figure 3.
Data visualization
from experimental
group (role playing)
Control group Experimental group
Sample size 51 31
Pre-test 0.9461** 0.8679***
Post-test 0.9381** 0.9186**
Notes: N reduced due to subjects with an incomplete Pre- or Post-test. *po0.10; **po0.05; ***po0.01
Table I.
Test for
normality – ShapiroWilk normality
test (W )
6.0
5.0
4.0
3.0
2.0
1.0
0.0
Scores (0–6 scale)
Pre-test Post-test
Figure 2.
Data visualization
from control group
(Traditional lecture)
302
JRIT&L
12,3
With these results (Tables II–IV), the experiment is considered valid and H2 can be
validated for both groups so further analysis can proceed to evaluate H3, which is to
compare between the groups.
4.2 Comparison of learning methods
To directly evaluate H3 as to the comparison between learning methods, a two-sample
comparison assuming unequal variances was utilized. Since the previous analyses
confirmed that the control group and experimental group were equivalent prior to their,
respectively, learning activities and that learning did occur in both groups, Table V details
the analysis showing a statistically significant higher level of learning by the experimental
group ( po0.10).
Using this role playing applied learning activity, students in the experimental group
reported a 45 percent gain in learning in comparison to those attending traditional lectures.
To further evaluate this result based on the relatively small sample and moderate variation
Control group Experimental group
Mean 1.911 1.933
Variance 1.408 0.711
Observations 51 30
Hypothesized mean difference 0
df 76
t-Stat 0.0995
P(T⩽t) two-tail 0.9210
t critical two-tail 1.9917
Table II.
Comparison of
groups – pre-test
scores
Pre-test Post-test
Mean 1.911 2.923
Variance 1.408 1.459
Observations 51 51
Pearson correlation 0.582
Hypothesized mean difference 0
df 50
t Stat −6.603
P(T⩽t) one-tail 0.000
t critical one-tail 1.676
Table III.
Verification that
learning
occurred – control
group (traditional
lecture)
Pre-test Post-test
Mean 1.933 3.397
Variance 0.711 1.600
Observations 30 30
Pearson correlation 0.167
Hypothesized mean difference 0
df 29
t Stat −5.732
P(T⩽t) one-tail 0.000
t critical one-tail 1.699
Table IV.
Verification that
learning occurred –
experimental
group (role playing)
303
The
effectiveness of
applied
learning
as expected in any human-subjects experiment, the effect size should also be considered. A
standard Cohen’s d can be computed if the standard deviations of the two samples are
roughly the same and therefore assumed they are estimating a common population
standard deviation. The computed effect size for this experiment using Cohen’s d was 0.360,
yielding a relatively small effect size as do0.50 (Cohen, 1988):
Cohen’s d ¼ m1 m2
spooled
;
where spooled ¼ ð
ffiffiffiffiffiffiffiffiffiffiffiffiffiffi
s2
1 þs2
2
q
Þ=2.
14
12
10
8
6
4
2
0
0 0.5 1.5 2.5 3 3.5 4 4.5 5 1 2
Scores (0–6 scale)
12
5
8
11
6 6
3
0000
Count of students
Figure 4.
Histogram of student
learning for the
control group
(traditional lecture)
Count of students
10
9
8
7
6
5
4
3
2
1
0
0 0.5 1.5 2.5 3.5 4.5 2345
1
0 0
1
0
1
2
8
9
4 4
1
Scores (0–6 scales)
Figure 5.
Histogram of student
learning for the
Experimental Group
(Role Playing)
Control group (traditional lecture) Experimental group (role playing)
Mean 1.012 1.463
Variance 1.198 1.956
Observations 51 30
Hypothesized mean difference 0
df 50
t Stat −1.516
P(T⩽t) one-tail 0.068
t critical one-tail 1.676
Table V.
Comparison of
groups – gain in
knowledge
(post- minus
pre-test scores)
304
JRIT&L
12,3
However, the standard deviations of the two groups did differ slightly (1.094 and 1.398,
respectively) so the homogeneity of variance assumption is questionable and pooling the
standard deviations may not be appropriate. One solution is to insert the standard deviation
of the control group into the equation and calculate the Glass’ Δ (Glass et al., 1981), assuming
that the standard deviation of the control group is untainted by the effects of the experiment
and will therefore reflect more closely the true population variance. “The strength of this
assumption is directly proportional to the size of the control group. The larger the control
group, the more it is likely to resemble the population from which it was drawn” (Ellis, 2009).
With the Control Group containing n ¼ 51, the Glass Δ was 0.413, yielding a similar small
effect size as the Cohen’s d for this relatively small experiment:
Glass0 Δ ¼ m1 m2
scontrol
:
And finally, it should also be considered that the experimental group (n ¼ 30) was smaller
than the control group (n ¼ 51). Taking this into account, the Hedges’ g will account for
sample size differences by weighting the variances by their respective sample size (Hedges,
1981). Hedges’ g was 0.372, re-confirming again the small effect size:
Hedges0 g ¼ m1m2
sn
pooled
;
where sn
pooled ¼
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ð Þ n11 s2
1 þð Þ n21 s2
2
=ð Þ n1þn22
q
.
5. Recommendations and conclusion
From the analysis of this experiment comparing the learning benefits of traditional lectures
vs applied learning using role playing, it was first verified that there was a gain in
immediate knowledge in both the modes of learning, confirming H2(a/b). The goals of this
study were met as the analysis showed significantly greater gain in knowledge (post-test
verses pre-test for each student) for the role playing groups (1.463 average gain in score) as
compared to those exposed only to the traditional lecture (1.012 average gain in score). This
represents a valuable 45 percent improvement in learning scores. Therefore, it is determined
that the time investment in preparing and executing the role playing activity was of
significant value to the customer – the students.
5.1 Limitations
This research was conducted using a single type of Applied Learning – role playing – and
using a single discipline within a business school. In addition, the application was tested in
the graduate-level educational environment, so findings are not generalizable to other
student levels or fields of study. However, findings are in-line with various other studies that
reported similar benefits of applied learning techniques (Brown et al., 2018; Silva et al., 2018;
Bhattacharya and Neelam, 2018). Larger samples using a variety of applications should
continue to be explored.
5.2 Scope for future research
Applied learning has a vast potential in the fields of training and education. The scope for
this research has been limited to one topic area (SCM) and one tool of applied learning (role
playing). The learning assessment measures in this study (i.e. pre- and post-test) were
comprised of only a few multiple-choice questions, but real learning may be better captured
using a more extensive test including essay questions that are application-based. It is also
possible to conduct a qualitative assessment on the participants to study their subjective
experience of applied learning in comparison with traditional learning. Further, in the
305
The
effectiveness of
applied
learning
current research, the independent variable of gain in knowledge was considered, but other
variables like the learning experience, quality of learning, time efficiency of learning and the
capability to implement the concepts in business and managerial situations should also be
considered. And finally, long-term retention of knowledge gained in the classroom may be of
even greater importance to meeting the goals of education; longitudinal studies could be
conducted at the one-month, six-month and one-year milestones as done with many alumni
surveys that attempt to assess the value of the education as assessed after graduation.
Because the scope of research in the field of applied learning is vast, continued study and
implementation of applied learning techniques is highly recommended.
References
Accreditation Council for Pharmacy Education (n.d.), “Accreditation standards and guidelines for the
professional program in pharmacy leading to the doctor of pharmacy degree”, Version 2.0.,
available at: www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf (accessed March 5, 2016).
Alessi, S. and Trollip, S. (1985), Computer–Based Instruction: Methods and development, Prentice-Hall,
Upper Saddle River, NJ.
Alkin, M. and Christie, C. (2002), “The use of role-play in teaching evaluation”, American Journal of
Evaluation, Vol. 23 No. 2, pp. 209-218.
Barnabe, F. (2016), “Policy development and learning in complex business domains: the potentials of
role playing”, International Journal of Business and Management, Vol. 11 No. 12, pp. 15-29.
Bates, T. (2005), Teachnology, E-learning and Distance Education, Routledge, New York, NY.
Beck, L. and Chizhik, A. (2008), “An experimental study of cooperative learning in CS1”,
Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education, New
York, NY, pp. 205-209.
Bhattacharya, S. and Neelam, N. (2018), “Perceived value of internship experience: a try before you
leap”, Higher Education, Skills and Work-Based Learning, available at: www.emeraldinsight.
com/doi/full/10.1108/HESWBL-03-2018-0025
Boslaugh, S. and Watters, P.A. (2008), Statistics in a Nutshell: A Desktop Quick Reference, OReilly
Media, Sebastopol, CA.
Brown, C., Willett, J., Goldfine, R. and Goldfine, B. (2018), “Sport management internships:
recommendations for improving upon experiential learning”, Journal of Hospitality, Leisure,
Sport & Tourism Education, Vol. 22 No. 22, pp. 75-81.
Bryant, J. and Darwin, J. (2004), “Exploring inter-organisational relationships in the health service:
an immersive drama approach”, European Journal of Operational Research, Vol. 152 No. 3,
pp. 655-666.
Chang, Y., Chen, W., Yang, Y. and Chao, H. (2009), “A flexible web-based simulation game for
production and logistics management courses”, Simulation Modelling Practice and Theory,
Vol. 17 No. 7, pp. 1241-1253.
Christmas, D. (2014), “Authentic pedagogy: Implications for education”, European Journal of Research
and Reflection in Educational Sciences, Vol. 2 No. 4, pp. 51-57.
Christopher, M., Mena, C., Khan, O. and Yurt, O. (2011), “Approaches to managing global sourcing
risk”, International Journal of Supply Chain Management, Vol. 16 No. 2, pp. 67-81.
Chwif, L., Barretto, M. and Paul, R. (2000), “On simulation model complexity”, Proceedings Winter
Simulation Conference, IEEE, December, Orlando, FL, pp. 449-455.
Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences, Routledge Academic,
New York, NY.
Daly, A., Grove, S., Dorsch, M. and Fisk, R. (2009), “The impact of improvisation training on service
employees in a European airline: a case study”, European journal of marketing, Vol. 43 No. 3,
pp. 459-472.
306
JRIT&L
12,3
Dempsey, J., Lucassen, B., Haynes, L. and Casey, M. (1996), “Instructional applications of computer
games”, Technical Report No. ED394500, University of South Alabama, Mobile, AL.
Douglas, A., Miller, B., Kwansa, F. and Cummings, P. (2008), “Students’ perceptions of the usefulness of
a virtual simulation in post- secondary hospitality education”, Journal of Teaching in Travel and
Tourism, Vol. 7 No. 3, pp. 1-19.
Doymus, K., Karacop, A. and Simsek, U. (2010), “Effects of jigsaw and animation techniques on
students’ understanding of concepts and subjects in electrochemistry”, Educational Technology
Research and Development, Vol. 58 No. 6, pp. 671-691.
Elgood, C. (1997), Handbook of Management Games and Simulation, 6th ed., Gower Publishing,
Aldershot.
Ellis, P. (2009), “Effect size equations”, available at: www.polyu.edu.hk/mm/effectsizefaqs/effect_size_
equations2.html (accessed December 5, 2018).
Ferguson, D., Hanreddy, A. and Draxton, S. (2011), “Giving students voice as a strategy for improving
teacher practice”, London Review of Education, Vol. 9 No. 1, pp. 55-70.
Ferrero, G., Bichai, F. and Rusca, M. (2018), “Experiential learning through role-playing: enhancing
stakeholder collaboration in water safety plans”, Water, Vol. 10 No. 2, pp. 227-237.
Glass, G., Smith, M. and McGaw, B. (1981), Meta-Analysis in Social Research, Sage Publications,
Beverly Hills, CA.
Gooden, D., Preziosi, R. and Barnes, F. (2009), “An examination of Kolb’s learning style inventory”,
American Journal of Business Education, Vol. 2 No. 3, pp. 57-62.
Hedges, L. (1981), “Distribution theory for glass’s estimator of effect size and related estimators”,
Journal of Educational Statistics, Vol. 6 No. 2, pp. 107-128.
Hmelo- Silver, C. and Barrows, H. (2006), “Goals and strategies of a problem-based learning facilitator”,
The Interdisciplinary Journal of Problem-Based Learning, Vol. 1 No. 1, pp. 21-39.
Howell, J. (1991), “Using role-play as a teaching method”, Teaching Public Administration, Vol. 12
No. 1, pp. 69-75.
Hudson, J. (2009), Pathways Between Eastern and Western Education, Information Age Publishing Inc,
Charlotte, NC.
Hui, F. and Koplin, M. (2011), “The implementation of authentic activities for learning; a case study in
finance education”, E-Journal of Business Education and Scholarship of Teaching, Vol. 5 No. 1,
pp. 59-72.
Husbands, C. and Pearce, J. (2012), What Makes Great Pedagogy? Nine Claims from Research. Research
and Development Network National Themes: Theme One, National College for School
Leadership, Nottingham.
James, M. and Pollard, A. (2011), “TLRP’s ten principles for effective pedagogy: rationale, development,
evidence, argument and impact”, Research Papers in Education, Vol. 26 No. 3, pp. 275-328.
Johnson, D. and Johnson, R. (2005), “New developments in social interdependence theory”, Genetic,
Social, & General Psychology Monographs, Vol. 131 No. 4, pp. 285-358.
Johnson, D. and Johnson, R. (2009), “An educational psychology success story: Social interdependence
theory and cooperative learning”, Educational Researcher, Vol. 38 No. 5, pp. 365-379.
Katsaliakia, K., Mustafeeb, N. and Kumar, S. (2014), “A game-based approach towards facilitating
decision making for perishable products: an example of blood supply chain”, Expert Systems
with Applications, Vol. 41 No. 9, pp. 4043-4059.
Kilic, D. (2008), “The effects of the jigsaw technique on learning the concepts of the principles and
methods of teaching”, World Applied Sciences Journal, Vol. 4 No. 1, pp. 109-114.
Kim, M. and Hannafin, M. (2011), “Scaffolding 6th graders’ problem-solving in technology-enhanced
science classrooms: a qualitative case study”, Instructional Science: An International Journal of
the Learning Sciences, Vol. 39 No. 3, pp. 255-282.
307
The
effectiveness of
applied
learning
Klein, P. (2003), “Rethinking the multiplicity of cognitive resources and curricular representations:
alternatives to ‘learning styles’ and ‘multiple intelligences”, Journal of Curriculum Studies,
Vol. 35 No. 1, pp. 45-81.
Kotzab, H., Teller, C., Bourlakis, M. and Wünsche, S. (2018), “Key competences of logistics and SCM
professionals – the lifelong learning perspective”, Supply Chain Management: An International
Journal, Vol. 23 No. 1, pp. 50-64.
Kovacs, Z. and Pato, B. (2014), “Jobs and competency requirements in supply chains”, Procedia – Social
and Behavioral Sciences, Vol. 109 No. 8, pp. 83-91.
Lambert, D., Knemeyer, A. and Gardner, J. (2004), “Supply chain partnerships: model validation and
implementation”, Journal of Business Logistics, Vol. 25 No. 2, pp. 21-24.
Lambert, D., Knemeyer, A.M. and Gardner, J. (2010), Building High Performance Business Relationships,
Supply Chain Management Institute, Sarasota, FL.
Lewis, M. and Maylor, H. (2007), “Game playing and operations management education”, International
Journal of Production Economics, Vol. 105 No. 1, pp. 134-149.
Li, K., Hall, M. and Bermell-Garcia, P. (2017), “Measuring the learning effectiveness of serious gaming
for training of complex manufacturing tasks”, Simulation & Gaming, Vol. 48 No. 6, pp. 770-790.
McCarthy, J. and Anderson, L. (2000), “Active learning techniques versus traditional teaching styles:
two experiments from history and political science”, Innovative Higher Education, Vol. 24 No. 4,
pp. 279-294.
Martocchio, J. and Webster, J. (1992), “Effect of feedback and cognitive playfulness on performance in
microcomputer software training”, Personnel Psychology, Vol. 45 No. 3, pp. 553-578.
Miller, K., Hill, C. and Miller, A. (2016), “Bringing lean six sigma to the supply chain classroom: a
problem-based learning case”, Decision Sciences Journal of Innovative Education, Vol. 14 No. 4,
pp. 382-411.
Niemi, R., Heikkinen, H. and Kannas, L. (2010), “Polyphony in the classroom: reporting narrative action
research reflexively”, Educational Action Research, Vol. 18 No. 2, pp. 137-149.
Paul, R. and Ponnam, A. (2018), “Teaching customer experience quality and its significance in retail
management: a role playing game using Chinese Puzzle ‘Tangram’ ”, Decision Sciences Journal of
Innovative Education, Vol. 16 No. 2, pp. 126-139.
Persson, F. and Araldi, M. (2009), “The development of a dynamic supply chain analysis tool –
integration of SCOR and discrete event simulation”, International Journal of Production
Economics, Vol. 121 No. 2, pp. 574-583.
Phillips, J. (2005), “Strategies for active learning in online continuing education”, Journal of Continuing
Education in Nursing, Vol. 36 No. 2, pp. 77-83.
Preziosi, R. and Alexakis, G. (2011), “A comparison of traditional instructional methods and
accelerated learning methods in leadership education”, International Leadership Journal, Vol. 3
No. 1, pp. 79-89.
Qiu, T., Liu, H. and Yin, E. (2018), “Learners’ experiences on role-playing collaborative learning
supported by ELS: a case study of virtual company program”, International Conference on
Blended Learning, Springer, Cham, pp. 275-286.
Quinn, C. (1996), “Designing an instructional game: reflections on quest for independence”, Education
and Information Technologies, Vol. 1 Nos 3-4, pp. 251-269.
Rao, D. and Stupans, I. (2012), “Exploring the potential of role-play in higher education: development of
a typology and teacher guidelines”, Innovations in Education and Teaching International, Vol. 49
No. 4, pp. 427-436.
Rhodes, T. (2007), “Accelerated learning for what?”, Peer Review, Vol. 9 No. 1, pp. 9-12.
Robinson, C. and Taylor, C. (2007), “Theorizing student voice: values and perspectives”, Improving
Schools, Vol. 10 No. 5, pp. 5-17.
Ruey, S. (2010), “A case study of constructivist instructional strategies for adult online learning”,
British Journal of Educational Technology, Vol. 41 No. 5, pp. 706-720.
308
JRIT&L
12,3
Ruhanen, L. (2005), “Bridging the divide between theory and practice: experential learning approaches
for tourism and hospitality management education”, Journal of Teaching in Travel and Tourism,
Vol. 5 No. 4, pp. 33-51.
Sauser, W.I. Jr and Sims, R.R. (2018), “Showing business students how to contribute to organizational
cultures grounded in moral character”, in Khosrow-Pou, M., Clarke, S., Jennex, M., Becker, A. and
Anttiroiko, A. (Eds), Business Education and Ethics: Concepts, Methodologies, Tools, and
Applications, IGI Global, Hershey, PA, pp. 485-507, available at: www.igi-global.com/pdf.aspx?
tid=186560&ptid=179832&ctid=15&t=Editorial%20Advisory%20Board
Schwartzman, R. and Henry, K. (2009), “From celebration to critical reflection: charting the
course of scholarship in applied learning”, Journal of Applied Learning in Higher Education,
Vol. 1, pp. 23-25, available at: https://files.eric.ed.gov/fulltext/EJ1188538.pdf
Serbessa, D. (2006), “Tension between traditional and modern teaching: learning approaches in
Ethiopian primary schools”, Journal of International Cooperation in Education, Vol. 9 No. 1,
pp. 123-140.
Shapiro, S. and Wilk, M. (1965), “An analysis of variance test for normality (complete samples)”,
Biometrika, Vol. 52 Nos 3/4, pp. 591-611.
Silcock, P. and Brundrett, M. (2001), “The management consequences of different models of teaching
and learning”, in Middlewood, D. and Burton, M. (Eds), Chapter 3, Managing the Curriculum,
Paul Chapman, London, pp. 35-54.
Silva, P., Lopes, B., Costa, M., Melo, A.I., Dias, G.P., Brito, E. and Seabra, D. (2018), “The million-dollar
question: can internships boost employment?”, Studies in Higher Education, Vol. 43 No. 1,
pp. 2-21.
Sogunro, O. (2004), “Efficacy of role-playing pedagogy in training leaders: some reflections”, Journal of
Management Development, Vol. 23 No. 4, pp. 355-371.
Tabak, F. and Lebron, M. (2017), “Learning by doing in leadership education: experiencing
followership and effective leadership communication through role-play”, Journal of Leadership
Education, Vol. 16 No. 2, pp. 199-212.
Ten Eyck, R. (2011), “Simulation in emergency medicine training”, Pediatric Emergency Care, Vol. 27
No. 4, pp. 333-341.
Tobin, K. and Tippins, D. (1993), “Constructivism as a referent for teaching and learning”, in Tobin, K.
(Ed.), The Practice of Constructivism in Science Education, American Association for the
Advancement of Science, Washington, DC, pp. 3-21.
Tran, V. and Lewis, R. (2012), “Effects of cooperative learning on students at Giang university in
Vietnam”, International Education Studies, Vol. 5 No. 1, pp. 86-99.
Westrup, U. and Planander, A. (2013), “Role-play as a pedagogical method to prepare students for
practice: the students’ voice”, Högre Utbildning, Vol. 3 No. 3, pp. 199-210.
Yang, J.C. and Quadir, B. (2018), “Effects of prior knowledge on learning performance and anxiety in an
English learning online role-playing game”, Journal of Educational Technology & Society, Vol. 21
No. 3, pp. 174-185.
Yardley-Matwiejczuk, K. (1997), Role–Play: Theory & Practice, Sage, London.
Zain, Z., Subramaniam, G., Rashid, A. and Ghani, E. (2009), “Teaching students’ performance and
attitude”, Canadian Social Science, Vol. 5 No. 6, pp. 92-102.
Zhang, D. and Nunamaker, J. (2003), “Powering e-learning in the new millennium: an overview of
e-learning and enabling technology”, Information Systems Frontiers, Vol. 5 No. 2, pp. 207-218.
309
The
effectiveness of
applied
learning
Appendix
Note: Variations in question order, response order, and numerical examples were changed
between assessments
Source: Subject matter reference is Lambert et al. (2010)
Corresponding author
Timothy Pettit can be contacted at: <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>
For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>
310
JRIT&L
12,3

“”
### Summary of “The effectiveness of applied learning: an empirical evaluation using role playing in the classroom”

#### Authors:
Harneel Acharya, Rakesh Reddy, Ahmed Hussein (National University, San Diego)
Jaspreet Bagga (CGI Group, San Diego)
Timothy Pettit (Department of Management and Marketing, National University)

#### Abstract:
The study aims to evaluate the effectiveness of role-playing as an applied learning technique compared to traditional lecture methods in enhancing classroom experiences.

#### Methodology:
The study employed a pre-test/post-test design with several control and experimental groups. The subjects were graduate MBA students in a traditional classroom setting.

#### Key Findings:
- The experimental group (role-playing) showed a significant 45% higher gain in knowledge compared to the control group (traditional lecture).
- Both learning techniques improved knowledge, but role-playing was significantly more effective.

#### Research Limitations:
- Limited to one educational discipline and a single role-playing activity.
- The impact on long-term knowledge retention was not studied.

#### Practical Implications:
- Educators should incorporate more applied learning activities such as role-playing to enhance knowledge gain and potentially improve long-term retention.

#### Originality/Value:
- The study uses a customized role-playing activity within a business curriculum, showing higher knowledge gain and enjoyment, encouraging lifelong learning.

#### Purpose:
To better prepare future supply chain leaders and managers by comparing applied learning techniques (role-playing) with traditional lecture methods.

#### Background:
The rise of globalization and technological advancements necessitates improved pedagogical methods to better prepare students for dynamic business environments, especially in supply chain management (SCM).

#### Literature Review:
- Authored support for active learning methods like role-playing, which enhance engagement and knowledge retention.
- Emphasis on the need for new pedagogical approaches amidst the exploding pace of change and knowledge.

#### Problem Statement:
Many workers lack hands-on skills due to insufficient coaching, mentoring, and applied learning opportunities. There’s a need for more effective learning solutions than standard classroom lectures.

#### Hypotheses:
1. Both traditional lectures and role-playing will lead to a gain in knowledge.
2. Role-playing will yield a higher gain in knowledge compared to traditional lectures.

#### Experiment Details:
- Conducted with 91 MBA students over 1.5 years across seven course sections.
- Control group (traditional lecture) and experimental group (role-playing) each received a pre-test, learning activity, and post-test.

#### Results:
- The role-playing group (experimental) had a greater knowledge gain (1.463 average) compared to the lecture group (1.012 average).
- The statistical analysis confirmed the significant advantage of the role-playing method.

#### Conclusion:
The study validates the effectiveness of role-playing in educational settings, highlighting a valuable improvement in knowledge gain over traditional lectures. Continued research and implementation of diverse applied learning techniques are recommended for enhancing educational outcomes.
criticism: this study is extremely limited
### Criticism: Limitations of the Study

While the study by Acharya et al. provides valuable insights into the effectiveness of role-playing as an applied learning technique, it has several limitations that restrict the generalizability and applicability of its findings.

#### 1. Limited Scope of Educational Discipline:
- Single Discipline Focus: The study is confined to a graduate MBA program, specifically exploring a course within the realm of supply chain management (SCM). The effectiveness of role-playing could vary greatly across different disciplines, making it hard to generalize these results to other fields such as science, technology, engineering, and humanities.
- Lack of Diverse Subject Matter: Only one learning activity related to SCM was tested, limiting the understanding of role-playing’s impact across a broad spectrum of subjects within business education itself.

#### 2. Single Applied Learning Technique:
- Narrow Focus on Role-Playing: The study examines only one type of applied learning, role-playing. Other applied learning methods, such as internships, case studies, simulations, and problem-based learning, could offer different levels of effectiveness. The conclusions drawn cannot confidently support applied learning broadly without comparing multiple techniques.

#### 3. Short-Term Knowledge Assessment:
- Immediate Knowledge Gain: The study measures knowledge gain immediately after the learning activities via pre-test and post-test scores. Long-term retention and the sustained impact of the knowledge gained through role-playing aren’t addressed, which are crucial for real-world skill application and enduring educational outcomes.
- No Longitudinal Studies: Without follow-up testing at later intervals (e.g., 1 month, 6 months, 1 year), there’s no insight into how well students retain information over time, a critical factor in evaluating the true effectiveness of any educational technique.

#### 4. Small Sample Size:
- Limited Participant Pool: With only 91 graduate students over seven course sections, the sample size is relatively small. The results may not be robust enough to account for variability across different student populations and learning environments.
- Uneven Group Sizes: The control group (n=59) is significantly larger than the experimental group (n=32), which could introduce
Type an input and press Enter
Hi there!
Run
Status code from OpenAI server
<Response [200]>
Parameters
▼
Built with Gradiologo