Cognitive Style and Mobile Technology in E-learning in Undergraduate Medical Education- A Randomized Controlled Trial of Otolaryngology-head and Neck Surgery
Phase: NA
Conditions: Medical Education
Sponsor: Chang Gung Memorial Hospital

1 
 
Supplement. The Original Trial Protocol and Statistical Analysis Plan for 
The Institutional Review Board, Chang Gung Medical Foundation, Taipei, 
Taiwan, ROC 
 
Protocol Title: Cognitive Style and Mobile Technology in E-Learning in 
Undergraduate Medical Education- A Randomized Controlled Trial of 
Otolaryngology-Head and Neck Surgery 
 
Protocol No.: NSC-105-2511-S-182A-006- 
IRB No.: 105-5290C 
Principal Investigator(s): Li-Ang Lee 
Co-Investigators: Cheng-Keng Chuang, Li-Jen Hsin, Shu-Ling Wang, 
Chung-Guei Huang, Hsueh-Yu Li, Chia-Hsiang Fu, Chung-Jan Kang, Wei-
Chieh Chao 
 
Original Protocol (Figure1): 
Protocol No.: NSC-104-2511-S-182-010- 
IRB No.: 103-6959B 
Principal Investigator(s): Li-Ang Lee 
Co-Investigators: Cheng-Keng Chuang, Li-Jen Hsin, Shu-Ling Wang, 
Chung-Guei Huang, Hsueh-Yu Li, Chia-Hsiang Fu, Chung-Jan Kang, Wei-
Chieh Chao 
Duration of Approval: From 2015/12/03 to 2017/12/02 
Approved Protocol: 2014/12/10 V.1 
Approved Informed Consent: 2014/12/10 Version 1 
Date of Approval: 2015/12/03, was approved by the Institutional Review Board 
(the “IRB”) of Chang Gung Medical Foundation on 2016/10/28. The IRB is 
organized and operates according to Good Clinical Practice and the 
applicable laws and regulations. 
 
Modified Protocol (Figure 2): (Add AttrakDiff2 questionnaire; sample size 
estimation of the validation study) 
Protocol No.: NSC-105-2511-S-182A-006- 
IRB No.: 105-5290C 
Principal Investigator(s): Li-Ang Lee 
Co-Investigators: Cheng-Keng Chuang, Li-Jen Hsin, Shu-Ling Wang, 
Chung-Guei Huang, Hsueh-Yu Li, Chia-Hsiang Fu, Chung-Jan Kang, Wei-
Chieh Chao 

2 
 
Duration of Approval: From 2016/10/28 to 2017/12/02 
Approved Protocol: 2014/12/10 V.1 
Approved Informed Consent: 2014/12/10 Version 1 
Date of Approval: 2016/10/28, was approved by the Institutional Review Board 
(the “IRB”) of Chang Gung Medical Foundation on 2016/10/28. The IRB is 
organized and operates according to Good Clinical Practice and the 
applicable laws and regulations. 
 
Chairman: Tsang-Tang Hsieh, M.D., Institutional Review Board, Chang Gung 
Medical Foundation 
 
 
Note: The study has been registered on the ClinicalTrials.gov (The study has 
been registered in the ClinicalTrials.gov).  

3 
 
Figure 1. Original IRB Certificate. 
 
 
 

4 
 
Figure 2. Modified IRB Certificate. 
 
 
 
Topics：Cognitive style and mobile technology in e-learning in undergraduate 
medical education- A randomized controlled trial of otolaryngology-head and neck 
surgery 

5 
 
Abstract 
 
Background: New designs of 6-year undergraduate medical education (UME) 
mainly include (1) integral curricula of body organ system, (2) multiple methods 
of clinical teaching and assessment, and (3) generalism in UME. UME is meant 
to enable graduates to become undifferentiated general physicians. 
Accompany with decreasing educational hours in the classrooms and hospital, 
essential but minor components of primary healthcare such as ophthalmology 
and otolaryngology–head and neck surgery (ORL-HNS) is disproportionately 
under-represented in UME. In Canada, substantial downstream effects on 
managing ORLHNS problems have been noted in family medicine residents. In 
order to improve learning insufficiency and enhance clinical competency 
without increasing extra-hours in the classrooms and hospitals, novel medical 
education stresses on enabling self-directory learning and increasing learning 
hours outside the classrooms. Accordingly, we hypothesize that innovations in 
educational technology can enhance the learning outcomes of ORL-HNS. 
Purposes: This study is aimed to (1) setup up a mobile technology in e-learning 
(M-TEL) that can provide instant, interactive and reliable information, (2) to 
determine whether M-TEL is an effective tool for the instruction of ORL-HNS, 
and (3) to compare effects of different cognitive styles on learning outcomes of 
MTEL with various modules of medical education. 
Material and Methods: This is a randomized controlled trial. Firstly, we will 
setup a e-learning platform of emergent ORL-HNS disorders with translating 
into an “app” function that can execute in mobile devices. We will enroll 10 
academic physicians and 10 junior residents for assistance in developing, 
evaluating, and improving the e-learning instruction. Secondly, we will recruit 
60 UME students without previous training in ORL-HNS to undergo the Group 
Embedded Figures Test to determine their cognitive styles such as field 
dependence or field-independence. After blinded randomization, students are 
instructed on two modules of emergent ORL-HNS disorders, using either a 
standard e-learning of text-figure PowerPoint show or an interactive multimedia 
module. Subjects are evaluated on emergent ORL-HNS disorders using a text-
based assessment and a multimedia assessment take place prior to and 
following instruction. 
Anticipating Outcome: This study can (1) establish a M-TEL of ORL-HNS that 
can deliver an innovatively mobile e-learning to supplement the deficiency of 
the classroom hours, (2) confirm M-TEL can enhance the efficiency of the 
instruction of ORL-HNS, and (3) understand differences in learning outcomes 

6 
 
of M-TEL with various modules of medical education between field dependence 
and filed independence using this platform. 
 
Keywords: cognitive style, e-learning, mobile technology, randomized 
controlled trial, otolaryngology-head and neck surgery. 
 
 
 
 

7 
 
Original Trial Protocol: 
Protocol No.: NSC-104-2511-S-182-010- 
IRB No.: 103-6959B 
 
Background 
 
Since 2013, there was an important revolution medical education in 
Taiwan: shortening to six years and moving internship to postgraduate level. 
New designs for 6-year program of medical education mainly include (1) 
integral curriculum for body organ system, (2) multiple methods of clinical 
teaching and assessment, and (3) generalism in undergraduate medical 
education (UME).1 The ultimate goal of UME (defined as the 5 or 6 years of 
medical school training) is meant to enable graduates with core knowledge and 
skills at the highest level of performance competency to become 
undifferentiated general physicians.2 Clinical problems associated with 
otorhinolaryngology–head and neck surgery (ORL-HNS) are frequently 
encountered, especially in primary care providers. Therefore, educating UME 
students is an extremely important privilege and responsibility given to all 
otolaryngologists associated with a medical school. However, the shortening of 
overall education hours in classrooms and hospitals, essential but minor 
components of primary healthcare such as ophthalmology and ORL-HNS is 
disproportionately under-represented in UME. This mismatch between this 
educational need and existing curricular delivery has been thought to result in 
substantial downstream effects on managing ORL-HNS problems in family 
medicine residents in Canada.3  
In order to improve learning insufficiency and enhance clinical 
competency without increasing classroom/hospital time, novel medical 
education lays stress on enabling self-directed learning and augmenting 
learning outside the classroom.4-9 Adequate use of different learning strategies 
is one of the most important prerequisites of academic success. The actual use 
of learning strategies is the result of an interaction between individual and 
situational variables and can change during the academic years.10 Medical 
educations must encourage autonomous learning behaviors among UME 
students to present a positive attitude towards learning and research. A series 
of novel teaching methods have been developed and validated using e-learning 
and simulation. Selected technologies may have a role in medical student 
teaching such as 3D computer-assisted instruction,11 interactive virtual 

8 
 
patient,12 and e-learning modules.13 Of note, web-based multimedia instruction 
provides learners with self-directed independent learning opportunities based 
on didactic material enhanced with multimedia features such as video and 
animations and can be used to replace other didactic events (e.g. lectures) or 
it may be provided in addition to other learning opportunities.8 However, e-
learning has limited value in teaching complex spatial anatomy to novice 
learners, but good value in teaching basic clinical knowledge and selected 
technical skills.14  
Mobile technology is gaining popularity in recent years, as a means of 
immediate interactive multimedia communication and access to the Internet. It 
represents the next natural frontier in the evolution of e-learning and we hereby 
call “mobile technology in e-learning (M-TEL)”. Utilizing M-TEL can result in 
greater educational opportunities for UME students while simultaneously 
enhancing faculty effectiveness and efficiency. However, institutional readiness 
for e-learning or M-TEL adoption ensures the alignment of new tools to the 
educational and economic context.15 Clinical teachers looking to use e-learning 
need to make sure that it will meet both their UME learners' needs and the 
program's needs, and it has to align to the contexts in which it is used. The 
absence of a proper dialogue regarding issues about expectations and beliefs 
for learners may put up a barrier to creating alignment and supportive 
professional relationshp.16 Even novel e-learning cannot completely replace 
traditional face-to-face lectures, because students indicate that they consider 
traditional teaching as the basis of their education.17 Besides, learning 
constructs were established for identifying learners’ learning styles who would 
most likely benefit from a problem-based or computer-assisted curriculum in a 
UME population.18A recent investigation found a relatively high (nearly 80%) 
prevalence of daily using the iDoc application (App) in primary care providers 
and they were particularly interested in ORL-HNS information within the 
limitations previously identified in medical school curricula.19 Fung14 and 
Lewis20 anticipated that over the next few years there will be a rapid emergence 
of M-TEL, many of which will have direct implications for teaching ORL-HNS. 
Accordingly, we can hypothesize that innovations in educational technology 
such as M-TEL can improve the learning outcomes of ORL-HNS. 
Prior to deployment into the UME curriculum, initial enthusiasm of M-TEL 
should be tempered with an objective approach to critical pedagogical 
evaluation. To our best knowledge, there are no known peer-reviewed 
publications specifically related to the use of M-TEL in teaching ORL-HNS to 
UME students. 

9 
 
Objectives 
 
This study aims: (1) setup up a M-TEL that can provide instant, interactive, 
and reliable information; (2) to determine whether M-TEL is an effective tool for 
the instruction of ORL-HNS; and (3) to compare effects of different cognitive 
styles on learning outcomes of M-TEL with various modules of medical 
education. 
 
 
Importance 
 
1. Development of an innovative M-TEL teaching tool of ORL-HNS that 
have been verified by academic otolaryngologists and junior 
residents. 
2. Validation of the novel M-TEL of ORL-HNS in UME students. 
3. Real understanding of the differences in performance on two e-
learning modules: text-figure PowerPoint show (PPS) and interactive 
multimedia (IM) module. 
4. Better recognition of the differences in performance on two cognitive 
styles: field dependence and field independence. Reasonable 
comprehension of the interaction of cognitive style and e-learning 
modules.  
5. Critically pedagogical evaluation and improvement of this educational 
innovation in order to implement this M-TEL in UME curricula to 
reform the current ORL-HNS education. 
 
 
Reviews of Literatures 
 
More recently, the smartphone and tablet featuring with faster processors, 
improved memory, and smaller batteries, can not only provide mobile 
communication but also advanced functions such app utilities. Embedded e-
learning in such mobile devices can affect our educational environments. For 
example, the European Society of Cardiology engaged to develop an e-learning 
platform to provide a very systematic 360o approach to ensure every participant 
will have the skills, knowledge, and professional attitude.21 Hardyman et al.19 
found a high prevalence of iDoc usage in house staffs, a project offering a 
smartphone library of medical textbooks to medical trainees and concluded that 

10 
 
the iDoc app was found to facilitate several aspects of medical training, 
including: (1) learning from observation, (2) transitioning from medical student 
to resident to independent practice, (3) interactive discussions with colleagues 
and peers, and (4) real-time rapid access to reliable information. The flipped 
classroom approach in a professional skills course using a mobile device has 
been rated more positively than the traditional classroom on many different 
characteristics despite no significantly improved performance as assessed by 
a written multiple-choice examination.22 Tablets can enhance students’ ability 
to develop and employ self-regulatory skills in a clinical context.23 An important 
point highlighted and worth developing is the difficulties associated with 
accurately comparing and assessing different medical apps for smartphones. 
With thousands of medical apps available, it is highly improbable that a clinician 
has a working knowledge of the complete range available and can evaluate its 
utility.24 Therefore, Lewis has proposed a systematic self-certification model for 
M-TEL apps including (1) information must be authoritative; (2) purpose of the 
website; (3) confidentiality; (4) information must be documented: referenced 
and dated; (5) justification of claims, (6) contact details, (7) financial disclosure, 
and (8) advertising policy.20 
The role of M-TEL, especially as it pertains to the UME learner, is evolving. 
Important factors to consider during development of this tool include: (1) 
knowledge base of the learner, (2) complexity and nature of the learning 
objectives, (3) learning style of the learner, (4) understanding the features and 
limitations of different technological genres, and (5) a team approach to module 
development.14 A previous study indicated that M-TEL in a medical educational 
course has the potential to reduce the need for face-to-face lectures. At the 
same time examination results are not impaired.17 Therefore, M-TEL should 
augment, but not replace, interactive discussion with colleagues and mentors 
in medical training.14 Furthermore, good learners’ readiness may imply that the 
instructional designer can rely on e-learning strategies and build the course 
upon them. However, according to the slightly lower scores in “motivation” and 
“online discussion” subscales, it is recommended to stress more on strategies 
that improve these two components.25 Accordingly, it is needed to test students’ 
readiness or cognitive style in more different degree programs to generalize the 
results. As learning management system could enhance UME students’ 
learning and gained more knowledge, M-TEL should allow teacher-led and 
student-led platforms to be integrated in the future for an enhanced student-
centered experience.26 Graff used the Cognitive Styles Analysis to determine 
The participant’s cognitive style and found verbalisers are typically superior at 

11 
 
working with verbal information and long page segmentation of web-based 
instructional system.27 The Group Embedded Figures Test (GEFT) was firstly 
applied to assess cognitive style and instructional materials for medical 
students in 1981.28 Initial results showed few effects favoring matching 
cognitive style with instructional materials. Field-independent students 
performed better than field-dependent ones in all of the subjects.29 Again, pre-
instructional determination of cognitive style may help the students to match 
their suitable instructional materials by themselves willingness. 
It is well recognized that medical students are required to retain vast 
amounts of medical knowledge on the path to becoming physicians. An idea M-
TEL can provide several ways of problem solving for UME students. Firstly, 
PowerPoint modules with case-based questions can significantly enhance 
learning in the short-term comparing with PowerPoint modules without case-
based questions.30 Secondly, a well-designed, integrated, and interactive e-
learning solution can be an efficient method for facilitating the application, 
integration, and contextualization of ORL-HNS anatomy and radiology.31 Thirdly, 
an adaptive computer supporting collaborative learning system to manage 
study sessions using instructional design and learning object principles can 
gather strong positive feedback from students and take an important step 
towards bringing information management tools to support study decisions and 
improving learning outcomes.32 Fourthly, game-based e-learning promises a 
higher motivation of learners by presenting funny contents in an interactive, 
rule-based and competitive way, and can be used as an effective teaching 
method for self-instruction.33 Finally, a multimedia web-based learning platform 
may be able to deliver audiovisual mnemonics designed to improve memory 
retention of medical sciences.34 
In UME, there remains a role for traditional teaching paradigms such as 
lectures, labs, and standardized patients; the choice of instructional type 
should be essentially tailored to the nature of the learning outcomes. Although 
M-TEL is broadly accepted and desirable by today’s medical students, this 
technology should be woven into the fabric of UME pedagogical principles in a 
sensible and careful way, only after empiric assessment. The development 
and implementation of M-TEL is the framework of standardized competency-
based learning objectives, common to all graduating medical students. 
 
 
 

12 
 
Materials and Methods 
 
1. 
Study Proposal and Reasons: 
 
1.1. 
Study design 
This study is a prospective, randomized, parallel-controlled, blinded trial 
that will be conducting from August 1, 2015 to July 31, 2017 at a university 
(Department of ORL-HNS, School of Medicine, College of Medicine, Chang 
Gung University, Taoyuan, Taiwan). Figure 3 shows the study protocol.  
 
Figure 3. Block Diagram of the Study Protocol 
 
System-design study (n = 20; 10 academic physicians and 10 junior 
residents) in which an e-learning platform of emergent ORL-HNS disorders with 
translating into an App that is executable on the mobile device will be designed. 
(A) Instructions of the most common 10 emergent ORL-HNS disorders 
according to our previously unpublished data will be collected. (B) Two different 
e-learning modules (visual-auditory text-figure PPS or an IM) will be established. 
(C) The App software of two e-learning modules will be programmed; and (D) 
Tests including a multiple-choice test and a multimedia situational test 
assessment will be set. The participants will assist our developing, evaluating, 
and improving the e-learning instruction.  
Validation study will be performed on 60 UME subjects without previous 

13 
 
ORL-HNS training by pretest-posttest and self-assessment of satisfaction using 
the randomly assigned M-TEL instructions. Prior to the validation study, all 
participants will be assessed by the GEFT test for determination of their 
cognitive style (field dependence or field independence). The flowchart diagram 
following the CONSORT 2010 guideline demonstrated our study design 
(Figure 4).35  
 
Figure 4. CONSORT Flowchart of the Validation Study 
 
 
After randomization, students are instructed on two modules of emergent 
ORL-HNS disorders, using either a PPS or an IM module. Subjects are 
evaluated on emergent ORL-HNS disorders using a text-based assessment 
(multiple-choice test) and a multimedia assessment (situational test) take place 
prior to and following instruction. Self-reported satisfaction with the study 
materials by the participants was obtained after examination. The interactions 

14 
 
of field dependence/independence, PPS/IM, and learners’ satisfaction will be 
investigated in order to improve the preliminary version of M-TEL. 
 
1.2. 
System-design study (the first year) 
 
1.2.1. Instruction Collection 
We will arrange a ADDIE team for instruction design: 1) Analysis, 2) Design, 
3) Development, 4) Implementation, and 5) Evaluation.36 We will organize an 
ADDIE team for instruction design. For 5- or 6-year UME student who has no 
previous ORL-HNS knowledge and skill, we aim to deliver the training to 
management of emergent ORL-HNS dis- orders frequently encountered by 
general medical residents for helping them to fill a gap in professional 
knowledge and skills. 
Accordingly, we analyze our data bank. From August 21, 2004 to 
September 28, 2004, there were 300 consecutive patients visited the 
Department of Emergency, Linkou-Chang Gung Memorial Hospital, Taoyuan, 
Taiwan, and consulted the otolaryngologist (mean age 33.3±24.5 years [range, 
0-89]; 91 (30%) children and 209 (70%) adults; 180 (60%) men and 120 women 
(40%). We selected the 10 most common emergent ORL-HNS disorders, in 
descending order based on consultation frequency, including foreign body 
(44.7%), epistaxis (9.3%), ear trauma (7.3%), acute otitis externa (7.0%), deep 
neck infection (4.7%), head and neck cancer and associated complications 
(4.0%), 
acute 
otitis 
media 
(3.7%), 
nasal 
trauma 
(2.7%), 
acute 
pharyngotonsillitis (2.7%), and sudden deafness (2.3%). 
We further categorize these disorders according to anatomic location. The 
mostly encountered ORL-HNS disorders are listed in descending order of the 
prevalence: throat, ear, nose problems, and head & neck problems. Despite 
emergent head & neck disorders such as tumor bleeding, upper airway 
obstruction and deep neck infection are uncommon (9%), they are frequently 
severe and lethal. We will develop four illustration courses (ear, nose, throat, 
and head & neck) covered the top 10 emergent ORL-HNS disorders and 10 
virtual patient cases. The content for each case was based on learning 
objectives already established for the ORL-HNS pre-clerkship curriculum. 
Twenty guest investigators including 10 academic physicians (5 ORL-HNS and 
5 non-ORL-HNS) and 10 junior residents (first- or second-year resident; 5 ORL-
HNS and 5 non-ORL-HNS) will assist our developing at this moment. 
 
1.2.2. Module Establishment 

15 
 
An overview of four courses, 10 disorders, and 10 cases will be provided. 
The guest investigators will assist the evaluation of these two models.  
 
1.2.2.1. 
PPS: Instructions and cases are constructed with integrated 
text, images/videos, and audio as classroom lectures and will 
be processing as a PPS format. In this module, the learners 
can only read the text, see the images, and hear the audio 
fixedly. 
1.2.2.2. 
IM: Instructions and cases are constructed with the use of 
hyperlink with integrated text, images, and audio. An 
interactive 
instruction 
illustrates 
the 
short-page 
text 
accompany with audio and demonstrates the figures or 
videos with audio according the choice of learners without 
time limitation. A typical case describes a patient who 
presents with specific concerns. UME students are then 
presented with options for the next most appropriate steps 
and are guided through appropriate diagnostic tests and 
management of the patient with the assistance of text, pop-
up windows, images, and audio. 
 
1.2.3. Application Programming 
We will need computer programmers to code our M-TEL platform to the 
Apple or Android app and to establish a web-based host computer that can 
provide an internet databank. Leaners using mobile devices can download the 
instructive materials from our host computer anytime and anywhere and upload 
their learning records to our data bank. This platform can also collect learner’s 
study profiles in order to improve our M-TEL system and UME students’ 
performance. At this moment, the guest investigators will try out system and 
feedback their experience to us. 
 
1.2.4. Test Setup 
Figure 7 demonstrates the Kirkpatrick’s four levels of evaluation include: 
level 1, reactions; level 2, learning; level 3, transfer; and level 4, results.37 We 
will set three multidimensional evaluations in our validation study: self-
assessment of satisfaction (level 1), multiple-choice test (level 2), and 
multimedia situational test (level 3). The guest investigators will answer the 
multiple-choice test and multimedia situational test to improve the questions. 
 

16 
 
1.2.4.1. 
Multiple-choice test: Questions are obtained from previously 
developed examinations of emergent ORL-HNS disorders 
and undergraduate final examinations. Questions that are not 
adequately addressed in both self-study materials were 
removed. Then a nine-question multiple-choice test will be set. 
The test is then uploaded onto the M-TEL platform such that 
students can answer online and submit the test once 
completed. 
1.2.4.2. 
Multimedia situational test: Virtual patient cases with specific 
emergent ORL-HNS disorders are newly developed. 
Situational questions that are not adequately addressed in 
both self-study materials were removed. Then a multiple-
choice test with nine situational questions will be set. The test 
is then uploaded onto the M-TEL platform such that students 
can answer online and submit the test once completed. 
 
1.3. 
Validation study (the second year) 
 
1.3.1. Participants 
For the relative small sample size of the present study, we analyzed all 
variables using non-parametric approaches. The sample size for this study was 
estimated using the primary outcome effects (mean posttest score) in two 
previously published studies.12,30 Using a two-tailed Wilcoxon signed-rank test 
for calculating the sample size (normal parent distribution; effect size, 0.8; type 
I error, 0.05; power, 80%), we got a sample size of 27 in each group at the 
minimum power of 0.80. For considering a 10% drop-out rate to fulfill the criteria 
of intention-to-treat analysis, we needed at least 30 participants to attend this 
study. 
 
1.3.1.1. The inclusion criteria of the present study are:  
1) 
Age > 20 years old & < 70 years. 
2) 
UME students (defined as the 5 or 6 years of medical school 
training). 
 
1.3.1.2. The exclusion criteria are: 
1) 
Previous ORL-HNS training; 
2) 
Declining to participate. 
 

17 
 
1.3.2. Randomization procedure 
Therefore, 60 participants are enrolled in the present study. Subjective 
assessments of GEFT are conducted after enrollment. The randomization 
procedure assures balanced design in subjective snoring severity and we 
stratified our subjects into two subgroups: ‘field dependence’ and ‘field 
independence’ subgroups. Computer-generated lists of random numbers are 
created using Random Number Generators of SPSS software for allocation of 
the participants and are stratified by center with a 1:1 allocation using a fixed 
block size of 6 (Rv. Uniform [0, 1]) in both subgroups. The allocation sequence 
is concealed from the blinded researcher (CGH) before instruction adhering to 
our computer-generated randomization protocol. Participants are randomly 
allocated to either IM or PPS group (Figure 4). Half of the patients are 
randomized to the IM group and the remaining patients are randomized to the 
PPS group. Subjects are evaluated on emergent ORL-HNS disorders using a 
multiple-choice test and a multimedia situational test take place prior to and 
following instruction. Self-reported satisfaction with the study materials by the 
participants was obtained one week after examination. 
 
1.3.3. The GEFT 
The GEFT test which is the most widely used version of pencil-and-paper 
tests in field dependence/ independence investigations.38 The GEFT has a 
relatively high Spearman-Brown reliability coefficient of 0.82 for their 
instrument.39 The GEFT is a 25-item test that requires participants to locate and 
trace simple geometric figures embedded within progressively more complex 
ones. This test has three sections. The first section is a practice section that 
contains seven simple embedded figures, and it is not scored. The time limit on 
this section is two minutes. The real task begins at the second set and into the 
third one, where the participants have to find the simple geometric figures inside 
two 9-itemed set within the time limit of five minutes for each. Based on the 
number of correct answers given by subjects, the scores on GEFT range from 
0 (the most field dependent) to 18 (the most field independent). Those who 
intend to rely on external cues are less able to find the simple figures (field 
dependence), and those who hinge on internal cues are abler to find figures 
(field independence).40 In this study, we will apply a Chinese version of the 
GEFT that GEFT scores ≤ 12 are considered as ‘field dependence’ and those 
with GEFT scores > 12 are regarded as ‘field independence.41 
 
 

18 
 
1.3.4. Outcomes 
1.3.4.1. The mean change in test scores before and after two instructive 
modules is the primary outcome measurement.  
1.3.4.2. GEFT score and self-assessment of satisfaction are the secondary 
outcome measurements. 
 
1.3.5. Statistical analysis: 
Data are analyzed using the SPSS 22.0 software (SPSS Inc., an IBM 
company, Chicago, IL, USA) and GraphPad Prism 5 software (GraphPad 
Software, Inc., San Diego, CA, USA). The Kolmogorov-Smirnov test is used to 
assess the normal distribution of variables. Statistical significance will be 
determined using the Student t test for continuous variables and the Mann-
Whitney U test or Wilcoxon rank test for skewed variables. Percentage (%) of 
change ([posttest value - pretest value]/[pretest value] × 100) in multiple-choice 
test as well as in multimedia situational test. Categorical variables are analyzed 
using the chi-square test or the Fisher’s exact test when sample sizes were 
small. Two-sided P values < 0.05 are considered statistically significant. 
 
 
Anticipated Works, Outcomes, and Effects 
 
1. 
Anticipated progression of the study 
1.1. 
We will conduct a 2-year study between August 1, 2015 and July 30, 
2017. I 
1.2. 
n the first year, we plan to establish an M-TEL platform in App format and 
invite 20 testing personnel (academic physicians in teacher’s aspects 
and junior residents in beginners’ aspects) to help developing and 
verification. 
1.3. 
In the second year, we plan to recruit 60 subjects to undergo pretest-
instruction-posttest 
experiments 
and 
correlate 
anthropological 
characteristics, cognitive style, M-TEL modules, test category, 
performances, and satisfactions. 
 
2. 
Anticipated possible problems and resolutions  
2.1. 
Although we investigate the performance of M-TEL by pretest and 
posttest without any invasive procedure, participants may decline to 
attend this study due to unfavorable allocation and instruction after 
randomization. Moreover, we aim to control the age, gender, and GEFT 

19 
 
ratios in the present study. Accordingly, we need more eligible 
candidates as anticipated. Fortunately, there are approximate 150 UME 
students in our university each year. However, we need a study assistant 
to obtain written ICFs and manage our experimental data. 
2.2. 
Despite e-learning is useful in many medical education studies, its use 
is still limited due to deficient resource of mobile technology in UME 
students, unstable internet connection in Gueishan district of Taoyuan 
city, and difficulty in executing the M-TEL App software on different 
mobile devises. In this study, we have to develop a M-TEL platform to 
simultaneous teaching, interaction, and assessment on tablets and 
smart phones. We need a team work to overcome this technical problem. 
Our investigators including academic physicians, expert in web-based 
learning, and medical technician who have many experiences in 
preparing instructive materials, teaching UME students, and assessing 
performance of medical learning. Nevertheless, we need a full-time 
study assistant to help us to process the computer programming. 
2.3. 
In view of the online safety issues involved in handling participants’ 
privacy, the M-TEK will be enabling to protect personal information from 
hacking. Our need expert helps in maintaining internet security. 
Otherwise, anonymous using of M-TEL will provide the highest safety 
but cannot enhance the learners’ long-term performance. 
 
3. Anticipated 
contributions 
to 
academic 
research, 
national 
development, and other clinical applications  
As mentioned above, the presence of a paucity of ORL-HNS learning in 
current UME curricula highlight the need to enhance the learning performance 
inside and outside the classrooms in order to enhance ORL-HNS knowledge 
and clinical skills for UME students and postgraduate general physicians. This 
study will set up the important M-TEL platform for self-directed learning. 
Moreover, identification of personally cognitive style to choose a proper 
instructional material on the M-TEL platform can reform the ORL-HNS learnings 
in UME. Finally, the UME students may use the M-TEL anytime and anywhere 
without significant increasing their cognitive loads and improve learning 
outcomes. 
 
4. Anticipation of trainings for study researchers  
The medical teachers can be familiar with the design of the M-TEL system 
and increase their ability of handling e-learning to effectively blend instructional 

20 
 
design and learning objective principles. The professors of web-based learning 
can extend researches in the UME level. The computer programmers can train 
their ability to establish an innovative learning tool to deliver UME education. 
 
 

21 
 
Modified Protocol (Figure 2): (Add AttrakDiff2 questionnaire; sample size 
estimation of the validation study) 
Protocol No.: NSC-105-2511-S-182A-006- 
IRB No.: 105-5290C 
 
User Experience- the AttrackDiff2  
 
Global satisfaction score is measured using a visual analogue scale from 0 
to 10 as mentioned above. For more delicately evaluate the acceptance of 
technical innovations, we will assess “user experience” using the AttrakDiff2.42  
AttrakDiff2 is developed as a tool by Hassenzahl’s research group to be 
able to quantify attractive, identifiable, stimulating, and pragmatic qualities. The 
tool uses 4x7 anchor scales, in total 28 questions. The anchors are presented 
in the form of semantic differentials and a 7-point Likert scale is employed for 
rating the intensity of the items. The poles of each item are opposite adjectives 
(eg, “confusing-clear”, “unusual-ordinary”, “good-bad”). Table 4 demonstrate 
the AttakDiff2. Each of the mean values of an item group creates a scale value 
for pragmatic quality (PQ), hedonic stimulation (HQ-S), hedonic identification 
(HQ-I), and attractiveness (ATT).  
Attributes in the PQ group describe how easy the user finds it to work with 
the provided program or environment. Attributes belonging to HQ-S describe 
factors that encourage the personal growth of users and provide stimulation to 
give them the opportunity to enhance their knowledge and development. The 
attributes falling into the HQ-I category make it possible to identify the social 
impact that using a product can have for users, including the “messages” that 
are communicated by using the evaluated product. Last, the attributes of the 
ATT group depict the overall experience a product has to offer to its users, that 
is, its attractiveness. 
 
Sample Size Estimation of the Validation Study 
 
The sample size for this study has been estimated using the primary 
outcome effect (change in multiple-choice test scores) among six 
undergraduate medical students in the system-design study: increased 43±18 
[IM module] & 35±21 [PPS module]) in the pilot study. Using a two-tailed 
Wilcoxon signed-rank test for calculating the sample size (normal parent 
distribution; effect size, 0.41; type I error, 0.05; power, 80%) using G*Power 
3.1.9.2 software, we get a sample size of 26 in each group at the minimum 

22 
 
power of 0.80. For considering an approximate 10% drop-out rate to fulfill the 
criteria of intention-to-treat analysis, we need at least 30 participants to attend 
this study. Accordingly, we estimate that a total of 60 participants included in 
analysis is suitable for the validation study (Part II of the present study). For 
better characterization and comparison of primary outcome after team 
discussion, we decide to use “percentage change” in addition to “change” in the 
future study. 
 
 
 
 

23 
 
Reference 
 
1. 
Taiwan Medical Accreditation Council. Six-years program of medical education. 
http://www.heeact.edu.tw/ct.asp?xItem=14388&ctNode=332&mp=3, 
accessed 
6 
December 2014. 
2. 
American Association of Medical Colleges. Recommendations for clinical skills curricula 
for undergraduate medical education. Executive summary. Washington, DC: American 
Association of Medical Colleges; 2005:3. 
3. 
Glicksman JT, Brandt MG, Parr J, Fung K. Needs assessment of undergraduate 
education in otolaryngology among family medicine residents. J Otolaryngol Head Neck 
Surg 2008;37:668-675. 
4. 
Effective Use of Educational Technology in Medical Education. American Association of 
Medical Colleges (AAMC) Institute for Improving Medical Education. Washington 2007:3. 
5. 
Fasce HE, Ortega BJ, Pérez VC, Márquez UC, Parra PP, Ortiz ML, Matus O. Association 
between self-directed learning behaviors, socio-demographic and academic variables 
among medical students. Rev Med Chil 2013;141:1117-25. 
6. 
Mahmud W, Haroon M, Munir A, Hyder O. Self-directed learning and research attitudes 
among medical students. J Coll Physicians Surg Pak 2014;24:173-7. 
7. 
Wolff M, Wagner MJ, Poznanski S, Schiller J, Santen S. Not another boring lecture: 
Engaging learners with active learning techniques. J Emerg Med 2014 Oct 13. pii: S0736-
4679(14)00930-5. doi: 10.1016/j.jemermed.2014.09.010. [Epub ahead of print] 
8. 
Yavner SD, Pusic MV, Kalet AL, Song HS, Hopkins MA, Nick MW, Ellaway RH. Twelve 
tips for improving the effectiveness of web-based multimedia instruction for clinical 
learners. Med Teach 2014 Aug 11:1-6. [Epub ahead of print] 
9. 
Lanken PN, Novack DH, Daetwyler C, Gallop R, Landis JR, Lapin J, Subramaniam GA, 
Schindler BA. Efficacy of an internet-based learning module and small-group debriefing 
on trainees' attitudes and communication skills toward patients with substance use 
disorders: Results of a cluster randomized controlled trial. Acad Med 2014 Oct 7. [Epub 
ahead of print] 
10. 
Fabry G, Giesler M. Novice medical students: individual patterns in the use of learning 
strategies and how they change during the first academic year. GMS Z Med Ausbild 
2012;29:Doc56. 
11. 
Cottam WW. Adequacy of medical school gross anatomy education as perceived by 
certain postgraduate residency programs and anatomy course directors. Clin Anat 
1999;12:55-65. 
12. 
Kandasamy T, Fung K. Interactive internet based cases for undergraduate 
otolaryngology education. Otolaryngol Head Neck Surg 2009;140:398-402. 
13. 
Yeung JC, Fung K, Wilson TD. Development of a computer-assisted cranial nerve 

24 
 
simulation from the visible human dataset. Anat Sci Educ 2011;4:92-97. 
14. 
Fung K. Otolaryngology-head and neck surgery in undergraduate medical education: 
Advances and innovations. Laryngoscope 2014 Aug 14. doi: 10.1002/lary.24875. [Epub 
ahead of print] 
15. 
Frehywot S, Vovides Y, Talib Z, Mikhail N, Ross H, Wohltjen H, Bedada S, Korhumel K, 
Koumare AK, Scott J. E-learning in medical education in resource constrained low- and 
middle-income countries. Hum Resour Health 2013;11:4. 
16. 
van Roermund TA, Mokkink HG, Bottema BJ, van Weel C, Scherpbier AJ. Comparison 
of expectations and beliefs about good teaching in an academic day release medical 
education program: a qualitative study. BMC Med Educ 2014;14:211. 
17. 
Nkenke E, Vairaktaris E, Bauersachs A, Eitner S, Budach A, Knipfer S, Stelzle F. 
Acceptance of technology-enhanced learning for a theoretical radiological science 
course: a randomized controlled trial. BMC Med Educ 2012;12:18. 
18. 
Chapman DM, Calhoun JG. Validation of learning style measures: implications for 
medical education practice. Med Educ 2006;40:576-83. 
19. 
Hardyman W, Bullock A, Brown A, Carter-Ingram S, Stacey M. Mobile technology 
supporting trainee doctors’ workplace learning and patient care: an evaluation. BMC Med 
Educ 2013;13:6. 
20. 
Lewis TL. A systematic self-certification model for mobile medical apps. J Med Internet 
Res 2013;15:e89. 
21. 
Wilkinson E. The new European Society of Cardiology e-learning platform for cardiology 
education. Eur Heart J 2013;34:3389. 
22. 
Moffett J, Mill AC. Evaluation of the flipped classroom approach in a veterinary 
professional skills course. Adv Med Educ Pract 2014;5:415-425. 
23. 
Archbold Hufty Alegría D, Boscardin C, Poncelet A, Mayfield C, Wamsley M. Using 
tablets to support self-regulated learning in a longitudinal integrated clerkship. Med Educ 
Online 2014;19:10.3402/meo.v19.23638.  
24. 
Ozdalga E, Ozdalga A, Ahuja N. The smartphone in medicine: a review of current and 
potential use among physicians and students. J Med Internet Res 2012;14:e128. 
25. 
Changiz T, Haghani F, Nowroozi N. Are postgraduate students in distance medical 
education program ready for e-learning? A survey in Iran. J Educ Health Promot 
2013;2:61. 
26. 
Zakaria N, Jamal A, Bisht S, Koppel C. Embedding a learning management system into 
an undergraduate medical informatics course in Saudi Arabia: Lessons learned. Med 2.0 
2013;2:e13. 
27. 
Graff M. Learning from web-based instructional systems and cognitive style. Br J Educ 
Tech 2003;34: 407-18. 
28. 
Scott N, Smith DU, Rosenberg IK. Cognitive style and instructional materials for medical 

25 
 
students. J Med Educ 1981;56:565-71. 
29. 
Tinajero C, Paramo MF. Field dependence and academic achievement: A reexamination 
of their relationship. Br J Educ Psychol 1997;67:199-212. 
30. 
Mounsey A, Reid A. A randomized controlled trial of two different types of web-based 
instructional methods: one with case-based scenarios and one without. Med Teach 
2012;34:e654-8.  
31. 
Webb AL, Choi S. Interactive radiological anatomy eLearning solution for first year 
medical students: Development, integration, and impact on learning. Anat Sci Educ 
2014;7:350-60. 
32. 
Taveira-Gomes T, Saffarzadeh A, Severo M, Guimarães MJ, Ferreira MA. A novel 
collaborative e-learning platform for medical students - ALERT STUDENT. BMC Med 
Educ 2014;14:143. 
33. 
Boeker M, Andel P, Vach W, Frankenschmidt A. Game-based e-learning is more effective 
than a conventional instructional method: a randomized controlled trial with third-year 
medical students. PLoS One 2013;8:e82328. 
34. 
Yang A, Goel H, Bryan M, Robertson R, Lim J, Islam S, Speicher MR. The Picmonic(®) 
Learning System: enhancing memory retention of medical sciences, using an audiovisual 
mnemonic Web-based learning platform. Adv Med Educ Pract 2014;5:125-32. 
35. 
Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, 
Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines 
for reporting parallel group randomised trials. BMJ 2010 Mar 23;340:c869. 
36. 
Morrison GR, Ross SM, Kemp JE, Kalman H. Designing Effective Instruction, 7th Edition. 
Wiley Inc., Hoboken, NJ, U.S.A., 2013.  
37. 
Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs. The Four Levels. Berrett-
Koehler Publishers: San Francisco, CA, U.S.A., 2006. 
38. 
Witkin H., Oltman P, Raskin E, & Karp S. Embedded figures test: Manual for the 
embedded figures test. Palo Alto, CA: Consulting Psychologists Press, 1971. 
39. 
Chapelle C, Green P. Field independence/dependence in second-language acquisition 
research. Language Learning 1992;42:47-83. 
40. 
Rittschof KA. Field dependence–independence as visuospatial and executive functioning 
in working memory: implications for instructional systems design and research. Educ 
Technol Res Dev 2010;58;99-114. 
41. 
謝國忠 （2004）「認知型態與訓練方法對電腦態度、電腦自我效能與學習績效之影響」，
國立中央大學資訊管理所未出版之碩士論文(Traditional Chinese)。 
42. 
Hassenzahl M, Burmester M, Koller F. Der User Experience (UX) auf der Spur: Zum 
Einsatz von www.attrakdiff.de. In: Brau H, Diefenbach S, Hassenzahl M, Koller F, 
Peissner M, Röse K, editors. Usability Professionals. Stuttgart: German Chapter der 
Usability Professionals Association; 2008:78-82.