iafor

journal of education

technology in education
Volume 8 – Issue 2 – 2020
Editor: Michael P. Menchaca
Editor-in-Chief: Yvonne Masters

ISSN: 2187-0594

iafor
IAFOR Journal of Education: Technology in Education
Volume 8 – Issue 2 – 2020

IAFOR Publications

The International Academic ForumIAFOR Journal of Education: Technology in Education
EditorMichael P. Menchaca
University of Hawaii at Manoa, USA
Editor-in-Chief
Yvonne Masters, Australia

Published by The International Academic Forum (IAFOR), Japan
IAFOR Publications, Sakae 1-16-26-201, Naka-ward, Aichi, Japan 460-0008

Executive Editor: Joseph Haldane
Publications Manager: Nick Potts

IAFOR Journal of Education: Technology in Education
Volume 8 – Issue 2 – 2020
Published July 17, 2020

IAFOR Publications © Copyright 2020
ISSN: 2187-0594
ije.iafor.org

Cover Image: Cover Image: h heyerlein / Unsplash

IAFOR Journal of Education: Technology in Education
Volume 8 – Issue 2 – 2020
Edited by Michael P. Menchaca
Associate Editors:
Daniel Hoffman
De-Graft Johnson Dei

Foreword(Musings from the Editor-in-Chief)
Technology in Education is our second issue for 2020. The issue’s editorial team have worked
hard, in difficult global conditions, to work with reviewers and authors to bring this issue topublication. Many thanks go to the editor, Michael P. Menchaca, and his two Associate Editors,
Daniel Hoffman and De-Graft Johnson Dei. While we have a special issue, COVID-19:
Education Responses to a Pandemic, opening for submissions later this year, we need toacknowledge now that technology in education has been at the forefront of manyeducationalists’ minds. We look forward to receiving many different and thought-provoking
submissions later in the year.
On reflecting on what might be said about technology and its use in education, I was remindedof the following comment:
As far as technology itself and education is concerned, technology is basicallyneutral. It’s like a hammer. The hammer doesn’t care whether you use it tobuild a house or whether on torture, using it to crush somebody’s skull, thehammer can do either. (Noam Chomsky).
Technology, the “knowledge or use of the mechanical arts and applied sciences” (one definitionin The Australian Pocket Oxford Dictionary), has been in education for far longer than many
current students would acknowledge. So often the term “technology in education” has come tomean the use of computers and the affordances of the Internet. Probably the first use oftechnology in education was the realization that you could use a stick to scrawl on the groundinstead of using a finger with all the hazards that can entail. Rudimentary technology indeed,but technology nevertheless.
Please don’t get me wrong: I am a fan of technology. As a university lecturer I ran tutorials in
Second Life for my distance education students: but I used it with a purpose, developing a
sense of social presence which has been shown to be important for engagement. I didn’t makethe technology drive the education. Education is and always must be the driving force behind
any use of technology.
On those perhaps controversial musings, my thanks to the authors, the editor, the associate
editors, the publications manager, Nick Potts, and to all the reviewers for bringing this issue toyou, the readers.
Enjoy,
Yvonne Masters,
Editor-in-Chief

Editorial Advice
Preparing a submission to the IAFOR Journal of Education is more than writing about yourresearch study: it involves paying careful attention to our submission requirements. Different
journals have different requirements in terms of format, structure and referencing style, among
other things. There are also some common expectations between all journals such as the use ofgood academic language and lack of plagiarism. To assist you in reaching the review stage forthis or any other peer-reviewed journal, we provide the following advice which you should
check carefully and ensure that you adhere to.
1. Avoiding Plagiarism
Plagiarism is a practice that is not acceptable in any journal. Avoiding plagiarism is the cardinalrule of academic integrity because plagiarism, whether intentional or unintentional, ispresenting someone else’s work as your own. The IAFOR Journal of Education immediately
rejects any submission with evidence of plagiarism.
There are three common forms of plagiarism, none of which are acceptable:
1. Plagiarism with no referencing. This is copying the words from another source (article,
book, website, etc.) without any form of referencing.
2. Plagiarism with incorrect referencing. This involves using the words from anothersource and only putting the name of the author and/or date as a reference. Whilst not as
grave as the plagiarism just mentioned, it is still not acceptable academic practice.
Direct quoting requires quotation marks and a page number in the reference. This isbest avoided by paraphrasing rather than copying.
3. Self-plagiarism. It is not acceptable academic practice to use material that you havealready had published (which includes in conference proceedings) in a new submission.
You should not use your previously published words and you should not submit about
the same data unless it is used in a completely new way.
2. Meeting the Journal Aims and Scope
Different journals have different aims and scope, and papers submitted should fit the specificjournal. A “scattergun” approach (where you submit anywhere in the hope of being published)
is not sound practice. Like in darts, your article needs to hit the journal’s “bullseye”, it needs
to fit within the journal’s interest area. For example, a submission that is about building bridges,
will not be acceptable in a journal dedicated to education. Ensure that your paper is clearly
about education.
3. Follow the Author Guidelines
Most journals will supply a template to be followed for formatting your paper. Often, there will
also be a list of style requirements on the website (font, word length, title length, page layout,
and referencing style, among other things). There may also be suggestions about the preferred
structure of the paper. For the IAFOR Journal of Education these can all be found here:
https://iafor.org/journal/iafor-journal-of-education/author-guidelines/

4. Use Academic Language
The IAFOR Journal of Education only accepts papers written in correct and fluent English ata high academic standard. Any use of another language (whether in the paper or the referencelist) requires the inclusion of an English translation.
The style of expression must serve to articulate the complex ideas and concepts being presented,
conveying explicit, coherent, unambiguous meaning to scholarly readers. Moreover,manuscripts must have a formal tone and quality, employing third-person rather than firstperson standpoint (when feasible), placing emphasis on the research and not on unsubstantiatedsubjective impressions.
Contributors whose command of English is not at the level outlined above are responsible forhaving their manuscript corrected by a native-level, English-speaking academic prior to
submitting their paper for publication.
5. Literature Reviews
Any paper should have reference to the corpus of scholarly literature on the topic. A review ofthe literature should:
•
•
•
•
•

Predominantly be about contemporary literature (the last 5 years) unless you are
discussing a seminal piece of work.
Make explicit international connections for relevant ideas.
Analyse published papers in the related field rather than describe them.
Outline the gaps in the literature.
Highlight your contribution to the field.

Referencing
Referencing is the main way to avoid allegations of plagiarism. The IAFOR Journal of
Education uses the APA referencing style for both in-text citations and the reference list. If
you are unsure of the correct use of APA please use the Purdue Online Writing Lab (Purdue
OWL), – https://owl.english.purdue.edu/owl/resource/560/01/ – which has excellent examples
of all forms of APA referencing. Please note APA is used for referencing not for the generalformat of the paper. Your reference list should be alphabetical by author surname and include
DOIs whenever possible.
This short guide to getting published should assist you to move beyond the first editorial review.
Failure to follow the guidelines will result in your paper being immediately rejected.
Good luck in your publishing endeavours,
Dr Yvonne Masters
Editor-in-Chief, IAFOR Journal of Education

Table of Contents
From the Editor
Michael P. Menchaca

1

Notes on Contributors

4

Facilitating Online Reading Comprehension in Enhanced Learning
Environment Using Digital Annotation Tools
Ruhil Amal Azmuddin
Nor Fariza Mohd Nor
Afendi Hamat

9

Democratization of Education through Massive Open Online
Courses in Asia
Runchana Pam Barger

31

Understanding the Characteristics of English Language Learners’
Out-of-Class Language Learning through Digital Practices
Ali Dincer

49

Beyond the Basics: Adapting an Open Textbook to Accommodate a
Flipped Class
Ariana Eichelberger
Hong Ngo

69

Learning Management System Adoption in Higher Education Using
the Extended Technology Acceptance Model
Marissa R. Fearnley
Johnny T. Amora

91

Digital Skills at Divine Word University, Papua New Guinea
Iwona Kolodziejczyk
Philip Gibbs
Cecilia Nembou
Maria Rodina Sagrista

109

Teacher Candidate Reflection and Development ThroughVirtual Exchange
Chesla Ann Lenkaitis
Shannon M. Hilliker
Kayla Roumeliotis

127

Training Factors as Predictors of Students’ Self-Efficacy Beliefs forOnline Journalism PracticeJoseph Njuguna143

Developing an Open Educational Resource and Exploring
OER-Enabled Pedagogy in Higher Education
Beth Tillinghast

161

Reviewers

177

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

From the Editor
Welcome to the IAFOR Journal of Education’s inaugural Technology in Education issue.
While the Journal structure may be slightly altered at times, our commitment tointerdisciplinary, international, intercultural, quality manuscripts remains. As editor, I rely
heavily on an extensive group including the editor-in-chief, two associate editors, and anincredible team of reviewers. I am truly humbled by everyone’s dedication, hard work, andsupport, especially given the challenges brought on by the global pandemic. Given current
circumstances, technology becomes even more significant in facilitating collaboration in theworld. Although IAFOR has reserved a special topic issue for manuscripts specifically related
to the impact of COVID-19, technology, particularly in education, plays an important role inour collective progress.
The scope of Technology in Education is purposefully broad to ensure the greatest
representation of contemporary knowledge impacting all learning, formal and informal. The
journal’s simple aim to improve education requires broad representation in scope and extensive
comprehension in depth. To achieve such depth, one factor binds all published manuscripts,
firm theoretical foundations. As editor, my one piece of advice to prospective authors: connect
your study to sound theoretical or conceptual foundations.
Reflecting IAFOR’s commitment to diverse knowledge and international collaboration, thenine published studies encompass:
•
•
•
•
•

Five continents: Asia, Europe, Africa, North America, and Oceania;
Seven countries: Malaysia, the Philippines, Turkey, Rwanda, the United States,
Mexico, and Papua New Guinea;
Three studies focusing on open education: MOOCs; Open Educational Resources; andOpen Textbooks;
Three studies focusing on online learning: learning management systems; online
journalism; and a virtual language exchange;
Three studies focusing on general e-learning: informal language learning throughdigital media; digital annotations; and digital literacy.

Please find below a brief summary of each article.
In the first article, Ruhil Amal Azmuddin, Nor Fariza Mohd Nor, and Afendi Hamat employ
qualitative focus group and content analysis methods to investigate how digital annotation tools
facilitate reading comprehension. Their “Facilitating Online Reading Comprehension in
Enhanced Learning Environment Using Digital Annotation Tools” article examines how
Malaysian students in a public university rely on digital annotation tools to assist in theiroverall reading comprehension. Their research suggests that students will benefit from training
on how to approach annotation tasks in online environments.
Runchana Pam Barger, in her article, “Democratization of Education through Massive Open
Online Courses in Asia,” uses content analysis to examine the promise of Open Education todemocratize education in Asia. The study indicates that while MOOCs can promote social andeconomic mobility, barriers, such as language, culture, and lack of digital literacy and technicalknowledge, prevent underserved populations in particular from successfully pursuing a digital
education. Collaboration between governments, corporations, non-government organizations,1

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020and local communities is needed to provide the infrastructure and support necessary to promote
more equitable opportunities for learning.
The third article, “Understanding the Characteristics of English Language Learners’ Out-ofClass Language Learning through Digital Practices,” by Ali Dincer, draws on mixed-methods
to look at students’ use of digital tools outside the classroom for English language learning at
three state universities in Turkey. The study identifies the importance for teachers to linkclassroom teaching to outside digital tools such as social media, games, YouTube, andintelligent tutoring applications.
Ariana Eichelberger and Hong Ngo, in “Beyond the Basics: Adapting an Open Textbook to
Accommodate a Flipped Class,” utilize a mixed-methods approach to determine how
undergraduate Biology students at a public university in Hawaii perceive an Open Biology
textbook, including identification of design features, course satisfaction, and suggestions for
improvement. Design decisions made based on conceptual frameworks as well as student
feedback improved the text and the students’ overall experience.
In “Learning Management Systems Adoption in Higher Education Using the Extended
Technology Acceptance Model,” Marissa R. Fearnley and Johnny T. Amora employ structural
equation modeling to determine the influence of technology acceptance constructs among
teachers in using a Learning Management System at a private college in the Philippines.
Significant implications of the study include the importance of self-confidence and system
quality in technology adoption.
In the sixth article, Iwona Kolodziejczyk, Philip Gibbs, Cecilia Nembou, and Maria Rodina
Sagrista conduct regression analysis in a study of the level of digital skills among university
students in Papua New Guinea. Their study, “Digital Skills at Divine Word University, Papua
New Guinea,” suggests that operational and formal skills are necessary, but not sufficient, forhigher information skills. Compared to countries such as the Netherlands, the effects of thedigital divide mean first year students in Papua New Guinea are at a disadvantage and face asteep learning curve in attaining digital literacy, thus needing exposure to digital skills muchsooner.
Chesla Ann Lenkaitis, Shannon M. Hilliker, and Kayla Roumeliotis, through qualitative
analysis, examine the effects of a virtual exchange among teacher training candidates learningto teach English to Speakers of Other Languages in their article, “Teacher Candidate Reflection
and Development Through Virtual Exchange.” Teacher candidates at a U.S. university
partnered with English as a Foreign Language learners at a Mexican university to assist in
language learning. The study supports direct language collaboration through virtual exchange
as a contributor to teacher candidate success.
The eighth article, “Training Factors as Predictors of Students’ Self-Efficacy Beliefs for Online
Journalism Practice,” authored by Joseph Njuguna, employs correlation analysis to determine
the strength between training factors and students’ self-efficacy in five Rwandan universities.
The results show the importance of training in fostering students’ beliefs and confidence intheir capacity to perform online.
In the final article, “Developing an Open Educational Resource and Exploring OER-enabled
Pedagogy in Higher Education,” Beth Tillinghast relies on an interpretive qualitative approach
to explore the perspectives of staff and faculty developing an Open Textbook using OER-

2

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020enabled Pedagogy in a public university in the United States. Constructs including attitude,
performance expectancy, and facilitating conditions are found to be of critical importance in
OER development. Further, the study recommends a team approach to OER development.
I hope you enjoy reading these articles as much as I enjoyed reviewing them.
Michael P. Menchaca
Editor, IAFOR Journal of Education: Technology in Educationtech.editor.joe@iafor.org

3

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Notes on Contributors
Article 1: Facilitating Online Reading Comprehension in Enhanced Learning
Environment Using Digital Annotation Tools
Dr Ruhil Amal Azmuddin is a university lecturer in Centre for Modern Languages, Universiti
Malaysia Pahang. She has teaching experience spanning many years in renowned higher
education institutions such as International Islamic University Malaysia (IIUM) and
Limkokwing University. She obtained her PhD in English Language Studies from Universiti
Kebangsaan, Malaysia and a Master of Educational Studies from University of Newcastle,
Australia. Amal has published locally and abroad in various peer-reviewed and Scopus journals
in her fields of interest, especially in reading English as a Second Language (ESL), Extensive
Reading (ER), online reading strategies and teaching reading with technology. She has alsopresented at conferences locally and abroad in those areas. Her other writing experience
includes a contribution to a chapter in a book, a module and a book on ER.
Email: ruhilamal@ump.edu.my
Dr Nor Fariza Mohd Nor, an Associate Professor at the Center for Research in Language andLinguistics, Faculty of Social Sciences and Humanities, Universiti Kebangsaan Malaysia, is
the Chair of the Center. She received her BA (Hons) in Modern English Language Educationfrom Lancaster University, UK; MA in English for Specific Purposes from Warwick
University, UK; and PhD in Applied Linguistics from University of Malaya, Malaysia. Herarea of expertise and research interests are discourse studies (spoken discourse, e-discourse),
critical discourse analysis and e-learning. Nor Fariza has presented at conferences andpublished her works in ISI, Scopus and indexed journals. She has also co-edited several books.
Dr Afendi Hamat is an Associate Professor at the Center for Research in Language andLinguistics, Faculty of Social Sciences and Humanities, Universiti Kebangsaan Malaysia. His
research areas are technology-assisted language learning and learning systems design. He has
published various journal articles and books on his areas of research. His more recent work iscross-disciplinary in nature with collaboration in public health and health informatics.
Article 2: Democratization of Education through Massive Open Online Courses in Asia
Dr Runchana Pam Barger has a PhD in Cultural and Educational Policy Studies with a
specialization in Comparative International Education. She is an Assistant Professor of TESOL(Teaching English to Speakers of Other Languages) in the Applied Linguistics andInternational Education Department at Wheaton College in Illinois, USA. She also is theProgram Director of the English Partnership Programs at Wheaton College Graduate School.
She was a U.S. Fulbright Scholar researching and lecturing in the Faculty of Education at
Chiang Mai University in Thailand. Her research focuses on Massive Open Online Courses(MOOCs), technology, religion, gender, and cross-cultural studies in education. She is theauthor of Religious Influences in Thai Female Education (1889–1931).
Email: pam.barger@wheaton.edu

3
4

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Article 3: Understanding the Characteristics of English Language Learners’ Out-ofClass Language Learning through Digital Practices
Dr Ali Dincer has a PhD majoring in English Language Teaching. He is an Assistant Professor
at Erzincan Binali Yıldırım University, Erzincan, Turkey. He also serves as the Head of Foreign
Language Education in his faculty and Editor-in-Chief of Erzincan University Journal of
Education Faculty. His primary research interests focus on psychological factors in language
teaching including language learning motivation, autonomous language learning and classroom
engagement. He is currently conducting research on out-of-class autonomous language
learning with technology.
Email: adincer@erzincan.edu.tr
Article 4: Beyond the Basics: Adapting an Open Textbook to Accommodate a Flipped
Class
Dr Ariana Eichelberger is an Associate Specialist faculty member with the Department of
Learning Design and Technology at the University of Hawaii at Manoa. She has a PhD in
Education specializing in Educational Technology. Dr Eichelberger is the Instructional
Designer for the College of Education and leads its Instructional Support Group providing
faculty professional development and technology support. She also works with the College's
Distance Course Design and Consulting Group as an instructional designer developing online
courses and programs. Her research areas of interest include online teaching, online course andprogram development, technology integration and faculty professional development.
Email: ariana@hawaii.edu
Dr Hong Ngo has a PhD in Education specializing in Learning Design and Technology. Dr.
Ngo earned other advanced degrees in Educational Administration and Educational
Technology. She works as an instructional designer and currently serves as a senior manager
in the Academic Research of the Vietnam-USA Society English Centers. Her research interests
include instructional design, online learning ecologies, educational leadership, school violence
and family violence.
Email: hongngo@vus-etsc.edu.vn
Article 5: Learning Management System Adoption in Higher Education Using the
Extended Technology Acceptance Model
Marissa R. Fearnley is an Assistant Professor teaching science at the School ofMultidisciplinary Studies (SMS) of De La Salle-College of Saint Benilde (DLS-CSB) in
Manila, Philippines. She also serves as the Director of the Center for Instruction, Research, andCurriculum (CIRC), the College’s faculty development arm. She has a degree in Master of
Public Health and Master of Learning and Teaching. Her research interests include technologysupported teaching and learning and professional development in higher education.
Email: marissa.fearnley@benilde.edu.phJohnny T. Amora is the Officer-in-Charge of the Center for Learning and Performance
Assessment (CLPA) of DLS-CSB. Presently, he is the President of the Philippine Associationof Researchers and Statistical Software Users (PARSSU). He facilitates seminars andworkshops related to applied statistics and analytics. He holds a degree in Master of Applied
Statistics, with research interests in the applications of statistical modelling in studies related
to human sciences, academia, and business.

4
5

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Article 6: Digital Skills at Divine Word University, Papua New Guinea
Dr Iwona Kolodziejczyk is an Associate Professor and Director of the Centre for Learning
and Teaching at Divine Word University in Papua New Guinea. She is an active researcher;her research interests are in gender and technology, technology in higher education, andacademic integrity in technology-enhanced academic practices.
Email: ikolodziejczyk@dwu.ac.pg
Dr Philip Gibbs is Professor of Social Research and Deputy President at Divine Word
University, Papua New Guinea. He holds a Doctorate in Theology from the Gregorian
University, Rome, and an MBA from Divine Word University, Papua New Guinea. Recent
publications include Enga Tindi Pii. The Real World and Creative Imagination, in Sung Tales
from the Papua New Guinea Highlands; Using Mobile Phones to Track Anti-Witchcraft
Violence in Papua New Guinea (with Sarah Logan); and with Miranda Forsyth, Contagion ofViolence: The Role of Narratives, Worldviews, Mechanism of Transmission and Contagion
Entrepreneurs, International Journal for Crime, Justice and Social Democracy.
Email: pgibbs@dwu.ac.pg
Dr Cecilia Nembou is Professor of Higher Education Leadership and Management andPresident of the Divine Word University in Papua New Guinea. She holds a PhD in Operations
Research from the University of New South Wales, Sydney, Australia; Master of Science inOperations Research from the University of Sussex, Brighton, UK; Graduate Diploma in
Statistics from the Canberra College of Advanced Education, Canberra, Australia; andBachelor of Science in Mathematics from the University of Papua New Guinea, Port Moresby,
Papua New Guinea. Her research interests include systems modelling and quantitative analysis
for strategic planning, management and decision making, with particular focus on leadershipand management in higher education. She has held various academic and senior management
positions at the Divine Word University in Papua New Guinea, University of Wollongong in
Dubai, and University of Papua New Guinea.
Email: cnembou@dwu.ac.pg
Ms Maria Rodina Sagrista is a Spanish communications specialist. She holds an Erasmus
Mundus Master’s Degree in Women and Gender Studies from the University of Granada(Spain) and Utrecht University (The Netherlands) and an Honours Degree in Audiovisual
Communication from Ramon Llull University (Spain). She taught in the Communications
Department at Divine Word University, Papua New Guinea, and has served with the UN Relief
and Works Agency in Aman, Jordan and the International Rescue Committee in Mae Sot
Thailand.
Email: mariarsagrista@gmail.com
Article 7: Teacher Candidate Reflection and Development Through Virtual Exchange
Dr Chesla Ann Lenkaitis received her PhD from Temple University in Philadelphia,
Pennsylvania, USA. Before finishing her doctoral work and working at Binghamton
University, Dr Lenkaitis worked as a public high school teacher for over 8 years. She dedicatesherself to teaching Spanish and identifying the most effective ways to facilitate the learningand teaching of a second language (L2). Her areas of specialty include technology integration
into the L2 classroom. More specifically, she examines the use of virtual exchange and its
impact on L2 learning and teaching. Chesla Ann Lenkaitis has presented her research at various
international conferences and her work has been published in journals including Computer

5
6

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Assisted Language Learning, Journal of International and Intercultural Communication, andTeaching and Teacher Education. In addition, she co-edited the book Engaging teacher
candidates and language learners with authentic practice and is an Editor of the MEXTESOLJournal.
Email: chesla.ann.lenkaitis@gmail.com
Dr Shannon M. Hilliker received her Master’s in Teaching English to Speakers of Other
Languages (TESOL) and PhD at the University at Albany in Curriculum and instruction witha focus on language learning. She has taught elementary and university level English as a
Second Language (ESL) and teacher candidates since 1999. Dr Hilliker is an AssistantProfessor of TESOL at Binghamton University (Binghamton, New York, USA) where sheteaches linguistics and grammar classes, how to use technology in the classroom, andsupervises internships for both pre- and in-service teacher candidates. Her research interests
include rural education, teacher professional development, elementary ESL after school
programs, international student success, and online conversation and culture exchange. Her
main research agenda features studies related to virtual exchange between teacher candidates
as well as teacher candidates and second language learners.
Email: hilliker@binghamton.edu
Ms Kayla Roumeliotis is a graduate student in the Elementary Inclusive Education programat Teachers College, Columbia University (New York, New York). She is a recent graduate ofBinghamton University (Binghamton, New York), where she majored in psychology andcompleted two minors in education and Spanish. She is interested in how people acquire
languages, how teachers can be prepared to teach languages, and the use of technology tofacilitate learning.
Email: kr2788@tc.columbia.edu
Article 8: Training Factors as Predictors of Students’ Self-Efficacy Beliefs for Online
Journalism Practice
Mr Joseph Njuguna holds a Master’s degree in Communication Studies. He is currently
affiliated to the University of Rwanda as Research Associate as he finalizes his field research
towards a PhD in Mass Communication. Mr Njuguna’s research interests are in media
convergence, journalism education, media policy and regulation as well as ICT for
development.
Email: jnjuguna110@gmail.com
Article 9: Developing an Open Educational Resource and Exploring OER-Enabled
Pedagogy in Higher Education
Dr Beth H. Tillinghast has a PhD in Learning Design and Technology from the Universityof Hawaii at Manoa. She also holds an MS in Library Science and an MA in Educational
Technology. She has worked in education for over four decades, living and working incountries in Latin America, Southeast Asia, and the Middle East. She has recently retired fromthe University of Hawaii at Manoa Library as a librarian working in the area of scholarly
communication. Her research interests include areas of open educational practices with a focus
on Open Access and Open Educational Resources.
Email: betht@hawaii.edu

6
7

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

8

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Facilitating Online Reading Comprehension in Enhanced Learning Environment
Using Digital Annotation Tools
Ruhil Amal Azmuddin
Universiti Malaysia Pahang
Malaysia
Nor Fariza Mohd Nor
Universiti Kebangsaan Malaysia
Malaysia
Afendi Hamat
Universiti Kebangsaan Malaysia
Malaysia

7
9

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
The use of technology has now become an integral part of higher education in Malaysia becauseof its positive outcomes in teaching and learning. Despite its use, students are not able to fully
benefit from its full potential. This study investigated the use of digital annotation tools inInteractive Reading for Academic Disciplines to facilitate reading of English for Science andTechnology materials in a blended course among university students. Data were collected from12 students enrolled in English for Technical Communication in a public university on the East
Coast of Peninsular Malaysia. Data collection consisted of online annotations and focus groupinterviews. Analysis was descriptive and thematic, using SPSS and NVivo software,
respectively. Inter-rater reliability analysis was measured using Cohen Kappa reliability
analysis that yielded an almost perfect score, proving that the data were reliable. The findings
revealed that digital annotation tools facilitated reading comprehension among students in an
online reading environment. Annotation analyses revealed consistency in interview data when
students were able to paraphrase, extend and synthesize ideas. An implication from this study
suggests that training students’ regarding strategies on how to deal with annotation tasks should
be mandatory. This is integral for students to achieve comprehension of English for Scienceand Technology materials in an online environment.
Keywords: digital annotation tools, online reading, English for specific purposes, English forscience and technology

108

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Online materials have now become one of the main sources of knowledge for students,
especially in academic settings. With the spread of digital educational spaces such as blogs,
Online Discussion Forums (ODF), Wikis and Learning Management Systems (LMS), there isa rising demand for materials that can be read in digital formats. Within this context, it is theeducator’s role to equip students to read online and explore online features so that they canmake the most of the myriad of online resources (Gilbert, 2017) and also to enhance learnerautonomy. If students are not well equipped, they will face challenges reading online because
the materials change and distract readers with multimodal features (Cho & Afflerbach, 2017).
Hence, teaching students how to read effectively online is crucial as library materials have been
digitized to create e-books and online articles.
Research on online reading comprehension is expanding from traditional comprehension
models to include “the purposes that drive online reading, communicative outcomes of online
reading, and the continuously changing nature of the strategies, skills and dispositions that are
required during online reading comprehension” (Leu Zawilinski, Castek, Banerjee, Housand,Liu & O'Neil, 2007, p.5). Moreover, according to Murugaiah (2016), a number of ComputerAssisted Language Learning (CALL) technologies (for example electronic dictionaries andmultimedia annotations) are commonly employed in teaching and learning as they provide
assistance for learners in various ways. For example, there is a belief that using various online
annotation types is a good, supportive reading strategy for learners (Thoms & Poole, 2017).
Huang (2014) conducted a study to examine the effectiveness of online versus paper-based
reading strategy instruction on EFL learners’ reading comprehension on fifty-seven students.
The result revealed that the second most positive feedback was that the online highlighting
function of the annotation tool facilitated incidental vocabulary learning because the colourcoded highlight function helped students tracked the parts that require attention. They also used
the highlighting function to re-read for understanding. Ridder (2002, as cited in Huang, 2014)
reported that the highlighted or visible hyperlinks increase students’ willingness to consult
electronic dictionaries. In some cases, annotations or dictionary definitions are available forstudents but these are not readily accessible and students cannot use these tools to their fullpotential. Herold (2014) contends that students need to work with the online tools, which arevery useful, to learn at the highest level. Such online tools that can assist students in learning
include Digital Annotation Tools (DAT). Hence, to understand ways in which DAT facilitateteaching and learning, this study addressed the following research questions:
1. What are the types of annotations made by students in Interactive Reading for
Academic Disciplines (iREAD)?
2. How do annotation tools facilitate students reading of English for Science andTechnology (EST) online materials?
Previous studies have concentrated on the effectiveness of annotations on reading
comprehension. Some of these studies have concentrated on the use of annotations but havefailed to delineate the actual process involved in using annotations to assist students to reachhigher reading comprehension levels (Tseng, Yeh & Yang, 2015). However, far too little
attention has been paid to the importance of annotations in teaching and learning. There is stillmuch uncertainty that exists about the relation between annotation types and online reading
comprehension. If teachers are not able to identify exactly which annotation types are usefulfor reading comprehension, they will not be able to determine reading instructions to promotestudents’ comprehension levels. Thus, information obtained from what Malaysian English as a
Second Language (ESL) university students do while reading EST online materials would
contribute to the pedagogical aspect of English for Specific Purposes (ESP). This paper begins

11
9

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

with a literature review related to the study, then describes the methodology used, presents theresults, provides a discussion of these, and lastly draws conclusions.
Literature Review
Theoretical Discussion
One reading model that is important in the current study is the Interactive Reading Model(Rumelhart, 1977). According to Rumelhart (1977), reading combines both bottom-up and topdown processes. Interactive models recognize the simultaneous interaction of lower levelprocessing skills and higher-level cognitive skills. According to Duke and Pearson (2002),
when it comes to comprehension, visual representation will help students understand, organize
and remember. For example, highlighting information in a text is a supportive reading strategy
as it allows readers to remember what they read and extract important elements. Thus, someelements of annotations conform to reading comprehension theories because these elements
provide interactive reading opportunities that help students identify the key elements ofparagraphs (Lo, Yeh & Seng, 2013). The key elements in the interactive reading model are
important in the current study as students interact with texts by identifying parts of online
reading materials through the use of DAT.
Previous studies report students’ reading comprehension and their recall of information are
based on a student’s ability to recognize organizational structures (Lo et al., 2013, Gilbert,
2017; Ruhil Amal, Nor Fariza & Afendi, 2018). The ability to recognize organizational
structures allows students to create a mental representation of the information and to see thelogical links made by the author. Good readers can use text structures to retrieve the main ideas
and to help them memorize propositions gained from reading. There is a large volume of
published studies describing the role of teaching text structures to students (Duke & Pearson,
2002; Moss, 2004; Pardo, 2004). What is not yet clear is the impact of the types of annotations
that students make in identifying text structures in achieving comprehension. This indicates aneed to understand the various ways that DAT assist students in learning.
Blended Learning in Higher Institutions
The popularity of blended learning is increasing because it integrates online technology suchas learning management systems and platforms, making learning more interactive. Previous
studies recorded higher achievements and better attitudes toward learning when theeffectiveness of blended learning was compared to conventional teaching approaches (Thang,Wong, Noorizah, Rosniah, Najihah & Kemboja, 2012). Educators use blended learning as an
approach to teaching because of its potential to maximize learning and create a more efficientlearning environment. For example, blended learning is believed to increase communication
skills (Wang, Wooa & Zhao, 2009), improve critical thinking skills (Güzer & Caner, 2014),
and to support collaborative learning (Haryani, Wan Faezah & Nor Aini, 2012). In Malaysia,
50% of courses in higher institutions offer online courses because it is an effective means tocommunicate within the teaching and learning context in the current era (Norazah, Mohamed
Amin & Zaidan, 2011). Taken together, these studies reflect the benefits and need toincorporate blended learning in tertiary education contexts.
For example, in a blended learning English as a Foreign Language (EFL) environment, Lo etal. (2013) proposed an interactive approach in learning paragraph structure through the use of
an online annotation system, Paragraph Annotator. Paragraph Annotator includes threehighlight buttons; yellow for topic sentence, blue for controlling idea, and green for supporting
details. These functions allow readers to analyze paragraphs and use annotation tools to add

1012

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

personal ideas. Both a Cued Recall Test and a Free Recall Test were used to assess students’comprehension between experimental and controlled groups. The results revealed significantly
better performance in both tests. However, the author overlooks the fact that annotation types
contribute to performance. The results of this study suggest that online annotation technology
provides EFL students with a flexibility to interact with the text that would not be accomplishedin reading books alone.
Online Reading
With the development of technology, reading has shifted to using non-traditional media suchas reading information on the world wide web in the form of videos, pictures, sounds,
animations and hyperlinks known as online reading (Sung, Wu, Chen & Chang, 2015).
According to Sung et al. (2015), reading online often creates a “hypertextual” form of readingthat consists of nodes and hyperlinks. Reading this type of information requires the information
to be read according to the order of the nodes. Even though hypertexts are considered to be
non-linear, many still read them in a linear mode, merely transferring offline reading skills toonline reading. This concept has recently been challenged by Kiili & Leu (2019) who believein the complexity of the online reading and the challenges students face especially incollaborative online reading. There are specific strategies that are more appropriate for online
reading.
However, there are various strategies used to manage online information and to navigate
successfully by selecting links that are useful. This ability is required for successful online
reading as it allows the reader to understand and construct potential meanings (Ruhil Amal etal., 2018). As proposed by Cho and Afflerbach (2017), there are three levels of building
coherence in online reading: information comprehension; intertextual connection; andconstruction of reading paths. The first online reading strategy is related to traditional reading
strategies such as infering, analyzing text information, and evaluating whether the text fulfills
the reading objectives. The second online reading strategy, intertextual connection, refers tosynthesizing multiple online sources that require multiple-text linking strategies to criticallycompare, evaluate, and corroborate the information found in diverse documents, to identify
differences between sources, and to integrate content from different sources. The third strategy(construction of reading paths) refers to the construction of meaning through networked
information technology that requires careful evaluation and selection of links. What is not clear
is the impact of DAT on reading comprehension.
Digital Annotation Tools
More and more materials are becoming available as electronic documents, increasing the needfor mechanisms that allow online annotations because annotation mechanisms have thepotential to enhance reading online (Chiu-Jung & Pei-Lin, 2012). Annotation mechanisms suchas EDUCOSM (Nokelainen, Kurhila, Miettinen & Tirri, 2003) and PAMS (Su, Yang, Hwang& Zhang, 2010), allow students to annotate the same documents, and share and provide
feedback on the annotations (Tseng et al., 2015) to improve their reading comprehension.
Several researchers contend that annotation methods such as underlining unfamiliar
vocabulary, marking main ideas or key words, or adding notes for reflection improves reading
comprehension (Marshall, 1997). These annotation methods minimize the cognitive load of thereading process (Chun & Payne, 2004), encourage understanding by connecting information ina text (Abuseileek, 2012), and improve critical reading skills (Johnson, Archibald &Tenenbaum, 2010). Annotation methods also provide interactive reading opportunities to
identify key elements of paragraphs (Lo et al., 2013). Taken together, these are important
processes in reading academic online materials.

11
13

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Recent studies point out the importance of DAT to enhance teaching and learning because
annotated texts allow for better comprehension in comparison to those without annotations(Chen, Hwang, & Wang 2012). For example, annotation software such as RedPencil
encouraged students to be involved in their learning activities (Ahern, 2005). Students caneasily use RedPencil to submit assignments and to view others’ annotations or comments. It is
not surprising that annotations can help students’ comprehension in various ways. Despite this,much uncertainty still exists about the relation between annotation types and comprehension.
In another study of an annotation technology called HyLighter, social annotation was
examined. This study found students exhibited more critical-thinking skills than metacognitionand comprehension skills when working in groups compared to working individually (Johnson
et al., 2010). The study concluded that annotation, reflection and highlighting will not have asignificant impact if conducted alone.
The ability to highlight the main points becomes an important reading strategy that guides
students to achieve overall understanding of the online materials. Highlighting texts support
reading because of three reasons as hypothesized by Li, Tseng & Chen. (2016). Firstly,
highlighting is an encoding process that identifies key parts of a text. Secondly, highlighting
texts captures readers’ attention because they are able to recall texts easily compared to texts
that are not highlighted. Lastly, highlighting texts acts as visual signals that allow retrieval of
critical points during a reviewing process. Thus, annotations such as highlighting texts help
learners monitor their understanding of the text as they decipher the L2 reading (Thoms &Poole, 2017). Together, these studies provide important insights into the need to utilize DAT
to assist with the reading of online academic texts.
Methodology
The focus of the current study was to explore processes involved in reading EST materials inan online environment. This was conducted in an ESP course named English for TechnicalCommunication (ETC) that utilized DAT in an online reading platform named Interactive
Reading for Academic Disciplines (iREAD) for teaching and learning. Hence, a mixed method
approach was used as the research design because descriptive statistical data from students’use of DAT were used to support the data gained from the focus group interviews.
Participants
The study adopted a purposeful sampling method to obtain rich data. Purposeful samplingrefers to intentionally selecting a sample that explores the main concept being studied (Creswell& Plano 2011). Hence, a sample of 55 students from 614 students enrolled in English forTechnical Communication from various Science and Technology courses for Semester 1,
2016/17 were selected. Twelve students were then selected to be a part of the qualitative datacollection. There was an equal number of male and female students aged between 21 and 23
years old. All the students were taking the same English course; a compulsory level 2 English
course at the university. Each student was provided a consent form to be a part of the study.
The course is a blended course, where students met twice weekly for 2-hours tutorial and 2hours computer labs. Students were participating because they had to for this class. However,during the initial briefing, students were briefed on the research project and were given options
to move to other classes if they do not wish to participate. This is to ensure that they know theirinvolvement is entirely voluntary and there are no repercussions for them not getting involved.

12
14

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Instruments
Focus Group Interview protocols. One of the instruments used to collect data was throughFocus Group Interview (FGI) protocols. The objective was to collect a shared understandingthat focused on the current study and to generate data from the group. The interview protocol
was designed to gauge the opinions of students about the use of DAT in iREAD. The questions
were created by the researcher, which were then verified by experts in the field. The FGI
protocol consists of questions related to the use of DAT such as highlighting and writing down
annotation notes.
Online reading system: iREAD. An online reading platform called the Interactive Readingfor Academic Disciplines (iREAD) was used to obtain annotation data. The system wasdeveloped by Universiti Kebangsaan Malaysia experts (Nor Fariza, Afendi, Hazita, Noorizah& Vengadasamy, 2014) with various functionalities such as DAT, discussion forums, video
and audio features. Although the system consists of various online tools, the current study
highlights just one of the main features, the annotation tools. The annotation tools in iREAD
contain two features. The first feature allows students to highlight online reading materials withvarious colours such as yellow, red and green. The second feature allows students to write
notes, comments or information about the texts that were highlighted.
Online reading materials. Text selection was based on an ETC module utilized by students
during the semester. Each text was analyzed according to the Flesch Kincaid Readability Index
that generates the level of difficulty of the reading texts. This allows the researcher to determine
the suitability of the texts according to the students’ proficiency level. Table 1 summarizes the
topics covered during the two weeks.
Table 1: Topics in English for Technical Communication
Week
5
6

Topic
Process Explanations: How to
use a compass
Directional Process: At thefactory, from bean to bar

Flesch reading ease63.9 – Standard/ average
54.9 – Fairly difficult toread

In Week 5, the lesson required students to read a 397-word essay about how to use a compass.
The essay is written in a technical description format that consists of the vital elements (i.emain parts, dimensions) required in a complete write up. The essay consists of four paragraphs
describing how to use a compass and how compasses work. The readability index for this essay
was 63.9 – making it standard or average difficulty. The idea of selecting an average readability
text was to introduce a more difficult text in the following week. Hence, in Week 6, students
were required to read an essay entitled “At the factory: from bean to bar” with a readability
index of 54.9, which was fairly difficult to read. Here, students were required to highlight andsummarize in their own words the process of making chocolates.
Data Collection Procedures
Annotation data were collected in Week 5 and 6 during a 14-week semester. For the annotation
activities, students were required to highlight parts of the text and then rewrite in their own
words what each highlighted text meant. This involved four steps in using the digital annotation
tools: selecting the element to be highlighted; applying the corresponding highlighted color(yellow, green or red); adding comments to the highlighted text; and, clicking the save button.
At the same time, students were encouraged to use red highlighter to identify the topic sentence,

13
15

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

yellow to identify central ideas and green for supporting details. However, not all students usedthe colored highlighter accordingly. Figure 1 labels the process for using the digital annotation
tools.

Figure 1: Screenshot of annotation process
Each annotation consists of either a word, a sentence-length, or a paragraph, which were madeusing the DAT in iREAD, where each annotation was considered to be a unit of analysis. The
data were tabulated and described according to the themes created and illustrated in thefindings. FGI was conducted in Week 7, after the students completed activities related toTechnical Descriptions and Process Explanations. FGIs were conducted in two groups with 6
students in each group. The focus of the interview was to reflect on the use of DAT to facilitate
reading of EST online materials while completing the activities in iREAD. Data were analyzed
thematically.
Data Analysis
The study analyzed annotations using content analysis based on categories adapted fromMarshall (1997) as cited in Tseng et al. (2015) where Minf refers to marking information. Twoforms of Minf were categorized by the researcher as highlighted texts that identify keywords
or main ideas, and written notes that are rephrases of a keyword or main idea and summarized
ideas. The second type of annotations were Mvoc, which refers to marking vocabulary. Thus,
two different highlights identify different elements of the online text.
The FGI data were analysed using the six steps proposed by Creswell (2014), which includes
preparing and organizing the data, exploration of data through coding, creating themes,
representing themes through narratives, interpreting the results and, validating the accuracy offindings. Data were validated using Cohen Kappa inter rater reliability analysis based on thedeveloped themes. The calculation yielded a K value of 0.7, which indicated a substantial
agreement, showing that the data analysis had a high reliability.
Results
In order to understand types and ways annotation tools facilitated reading of EST online
materials, data are described thematically under two headings: identify paragraph structure,and improve understanding. The data described are gained from FGI and annotations made by
students.

14
16

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Identify paragraph structure
The following section discusses the use of iREAD’s annotation tools in identifying paragraphstructure when reading “How to use a compass”. The data are presented in the form of interview
excerpts (as depicted in words and phrases which are in bold) as well as a tabular format
through the activities students completed. The researcher highlighted the bold words that
identify the important points stated by students during the interviews. Students believe thatannotation tools assist them in identifying paragraph structure when reading a text. Theinterview excerpts in Table 2 demonstrate this point.
Table 2: Interview excerpts 1
Student
Student L
Student S

Student V
Student C

Interview excerpts
We can know the passage flow and also we can know how many main parts, howmany sub-parts according to the body. The body and the parts
Let’s say when we highlight, we can know what the paragraph is about like this part
we can talk about body then we can know the sub-part. This paragraph is talked
about body part, the function, dimension or description to us. I can know myreading progress
I can highlight the main idea, sub parts, main parts to let me recall back what I
have read. Oh, this is what source of description is. This is that and all that.
We do the activity for the technical description. So we want to know that which partyou want to highlight first. For example, you want to know the size, the material, thecolour, so we will focus on that

The interview excerpts in Table 2 are students’ claims that annotation tools assist them in
identifying parts of a paragraph structure such as “main parts”, “sub-parts”, “dimension”,“description” and “color” in understanding a technical description. In view of this, online
annotations made by students were analyzed in order to validate students’ ability to recognize
paragraph structure of a technical description as claimed by students in the interviews. The
annotation activity that was explored was based on the lesson in Week 5 (see Table 1). For thisparticular activity, none of the students highlighted texts on vocabulary (Mvoc). Table 3
summarizes the types of annotations made by students.
Table 3: Summary of annotations on Process Explanations
No

Student

1
2
3
4
5
6
7
8
9
10
11
12

Student E
Student T
Student L
Student G
Student S
Student K
Student V
Student J
Student C
Student R
Student H
Student A
Total

Highlighted
text (Minf)
9
8
4
8
0
4
7
1
6
9
4
6
66

15
17

Written notes(Minf)
6
8
4
8
0
4
7
1
6
8
4
0
56

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Based on Table 3, all of the annotations made by students were content-based annotations(Minf). This means all annotations highlighted and written were identification of paragraphstructure. There were 85% of written notes and only 15% were highlighted texts. This denotes
students were able to identify parts of a technical description. The following screenshots areexamples illustrating annotations made by students in identifying parts of a paragraph structure.
As shown in Figure 2, Student V was able to annotate parts of a paragraph structure. First,
Student V identified ‘orientation of parts’ as a key component in the introduction of a technicaldescription. Then, Student V identified “1st step” as part of how to use the compass. The otherannotations were also “2nd step” and “3rd step”. Student V was able to annotate 7 parts of aparagraph structure (see Table 3). These annotations are consistent with her claims in interview
excerpt: I can highlight the main idea, sub parts, main parts to let me recall back what I haveread. The bold words identify the important points stated by students during the interview.
Annotation No. 1:

Annotation No. 2:

Figure 2: Screenshot of annotation No. 1 and 2 by Student V

16
18

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Figure 3 demonstrates three examples in identification of paragraph structure by Student C.
Annotation No. 1:

Annotation No. 2:

Annotation No. 3:

Figure 3: Screenshot of annotations Nos. 1, 2 and 3 by Student C
Annotations Nos. 2 and 3 were both part of introduction of a technical description that consistsof “the shape and location of the object” as well as “the purpose of sub-part” as part of thecontent element in a technical description. Overall, Student C is consistent in being able toidentify paragraph structures as claimed in interview excerpt “…you want to know the size, thematerial, the colour, so we will focus on that. The bold words identify the important points
stated by students during the interview.

17
19

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Figure 4 demonstrates two examples in identification of paragraph structure made by Student E.
Annotation No. 1:

Annotation No. 2:

Figure 4: Screenshot of annotation no. 1 and 2 by Student E
Student E was able to annotate “how does a pole work” and “function of main part” as seen in
Figure 4. In total, he was able to make 12 annotations (see Table 3), in which 3 did not include
any written notes. This indicated that Student E understood parts of an essay on a technicaldescription and was able to identify technical description essay structure.
Improves Understanding
Another major finding was that reading online materials using DAT helps students comprehend
the texts better because they are able to paraphrase the text based on their own understanding(as depicted in words and phrases which are in bold). The researcher highlighted the bold words
that identify the important points stated by students during the interviews. This theme wasderived from interviews about “key stages of making chocolates”. The interview excerpts inTable 4 demonstrate this point. The researcher highlighted the bold words that identify theimportant points stated by students during the interviews.

18
20

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 4: Interview excerpts 2
Student
Student K
Student E
Student LInterview excerpts
We annotate in our way, so we understand better…we can just summarize all themain point then we can know about what the passage say about. It is more clear whenreading the text
For me the annotation tools is quite useful because when the key-point is there andhow they elaborate is very important and maybe we can rephrase it into anotherwords so that we can easily understand about that
I think it [annotation tools] is useful because it is easier for us to understand what
the text is going to talk about.

Based on Table 4, students K, E and L believe that DAT provide the opportunity to annotate
the information read online using their own linguistic knowledge to achieve comprehension.
This allows for the opportunity to elaborate the online information to reach comprehension. As
a result, learning becomes meaningful when understanding is attained.
According to students, using DAT when reading online materials improves understanding
because they were given the opportunity to highlight and then rewrite (through written
annotations) the ideas. This was important for students because the online reading materials
were complicated because of its S & T entities. This is further elaborated in Table 5. The
researcher highlighted the bold words that identify the important points stated by students
during the interviews.
Table 5: Interview excerpts 3
Student
Student E
Student H
Student C
Student A
Student K

Student R

Interview excerpt
I can read much more better than before this, because last time some of the words I
really can’t understand, throughout iREAD and those annotation by phrase and all that,
I can easily understand some of the new word that never seen before
While we are writing or going to present something. We have the ideas. The
manufacturing of the rice, the process. Those are all ideas that I now know andunderstandErm after we do some activity that we feel that we will more easy to understandabout what really want to do in the activity related about the activity and what not todo.
It makes me more understand the text and helps me analyze from sentence tosentence, assists me to find out main idea and explanations. Gives me an overall
understanding of the text that I readWe can, we use the tools so that we can easily know every details of the subject
about. Yeah, we can also know important, what the subject is mainly about the parts
or something. So when I do this it makes reading become easier to understandbecause I can organize it this way
Because previously when I read all these scientific passage, it is all in exam, it is all
for exam purpose. So that we just like read the question and find the answer, but
through this iREAD, we actually reading a passage, analyze the passage and thenwe understand in our own ways

All six students believe that DAT assisted them in understanding because it provides an outlet
to express comprehension of S & T information. As illustrated in Table 5, one possible
implication in achieving comprehension of EST online materials is when students were able tosummarize the main points. These responses illustrate the students’ ability to understand theonline materials because they were able to annotate the reading text in a way which is

19
21

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

comprehensible to them. For example, when students were able to paraphrase and elaborate thereading materials that relate to their academic background, they were also able to organize theirunderstanding accordingly. What this means is that topics such as processes that seem to becomplicated may be less complicated to understand. This is significantly important because
learning S & T in a SL can be rather challenging.
In order to validate the data obtained from the interviews, the study analyzed the annotations
made by students. This consists of Minf for both highlighted texts and annotation notes madeby students. The purpose is to confirm the students’ ability to understand the EST online
reading materials as claimed in the focus group interviews. Table 6 summarizes the annotations
made by students based on the activity in Week 6 (see Table 1).
Table 6: Summary of annotations on Directional Process
No
1
2
3
4
5
6
7
8
9
10
11
12

Student
Student E
Student T
Student L
Student G
Student S
Student K
Student V
Student J
Student C
Student R
Student H
Student A
Total

Highlighted text (Minf)
26
32
22
9
16
16
0
3
26
29
24
4
207

Annotation notes (Minf)
10
4
22
9
6
16
0
2
14
11
17
0
111

Table 6 reveals a total of 207 texts that were highlighted and 111 of which included written
notes. This means that 54% were the reproduction of main ideas. This shows that students
understand the texts as claimed in the FGIs. They reproduced the main ideas based on theirbackground knowledge and linguistic knowledge. This is complementary to Schema Theory(Carrell, Devine & Eskey, 1988), where background knowledge is matched with textual data.
Samples of annotations made by some of the students are shared in Figure 5.
Annotation No. 1:

20
22

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Annotation No. 2:

Figure 5: Screenshot of annotation no. 1 and 2 by Student R
Based on Figure 5, Student R was able to elaborate the first step of making chocolates whichis “roasting”. Throughout this activity, Student R was also able to identify each key stage
systematically. For example, he was able to identify the third stage which is “grounding”. As
seen in annotation no. 3, he paraphrased the idea through changing the sentence based on his
knowledge and interpretation. This is an indication of the use of support reading strategy(Anderson, 2003). As such, Student R was able to use proper reading strategy to facilitate
reading of EST online materials.
Annotation No. 1:

Annotation No 2:

Figure 6: Screenshot of annotation No.1 and 2 by Student L

21
23

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Figure 6 provides data on annotation notes made by Student L. Student L was able to identify‘supporting details’ and elaborate each supporting details. For example, in both annotations,Student L managed to write the supporting details based on his own interpretations. This isconsistent with his interview extract “I think it [annotation tools] is useful because it is easierfor us to understand what the text is going to talk about”. The bold words identify the important
points stated by students during the interviews. Thus, Student L was able to show his
understanding of EST online materials through the annotations that were made. Figure 7 areannotations made by Student T.
Annotation No. 1:

Annotation No. 2:

Annotation No. 3:

Figure 7: Screenshot of annotation numbers 1, 2 and 3 by Student T
Figure 7 illustrates three examples of annotation notes made by Student T. For annotation No.
1, Student T rephrased the idea of the first sentence in the paragraph to illustrate understandingof the new information read. In annotations Nos. 2 and 3, Student T was able to rewrite the22
24

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

sentence into his own understanding by interpreting the text based on his knowledge, thus
generating meaning. This was seen throughout the annotation activity that Student T
completed. According to Schema Theory, reading is an interactive and constructive process,where reading generates meaning through the process of relating the textual information toexisting knowledge (Carrell et al., 1988). Hence, annotations made by Student T illustrate that.
Discussion
The examples shared on written annotations produced by some of the students demonstrates
that students were able to identify paragraph structure as admitted by students in the FGIs. The
findings are consistent with previous research that concluded identifying paragraph structures
as an important entity to assist students in understanding of main ideas, thus develop reading
comprehension (Carrell et al., 1988; Meyer, Wijekumar, Middlemiss, Higley, Lei, Meier &Spielvogel et al., 2010). In relation to this, identification of paragraph structure is important forreading comprehension. This is because identifying paragraph structures leads to recognizing
and understanding main ideas of the paragraphs (Lo et al., 2013). In addition, utilizing DAT
assists students to understand these technical S & T materials, making reading academic texts
in English less complicated for ESL learners.
The findings revealed a close relation to Schema Theory where reading is considered aconstructive and active process that require readers to understand information based on
retrieving and applying previous knowledge. As hypothesized by Li et al (2016), highlighting
texts supports reading as it identifies key parts, recall information and act as visual signals. Allof these elements are important for ESL learners who are reading ESP texts in their SL.
Moreover, highlighting coupled with written annotations is powerful because it contextualizes
and synthesizes ideas. These findings play a complementary role in the Interactive Reading
Model (Rumelhart, 1977). It is believed that interactive reading provides opportunities to help
identify key elements of paragraphs which conforms to some elements of annotations. This
allow students to be explicitly ready to deal with syntactic and discourse features that are
generally used in the language of EST more easily with the use of DAT. In addition, dataobtained with regards to identifying paragraph structure and improved understanding were
triangulated with annotation analyses, confirming the validity and importance of DAT.
Finally, a number of important limitations need to be considered. Firstly, there were sometechnical constraints in using iREAD such as the inability for students to change the highlighted
colors twice. Instead, double highlighting was made. Secondly, DAT in iREAD should include
social annotation functions to integrate a more holistic approach to teaching and learning. This
would encourage collaborative learning among students. This could significantly increase amore interactive teaching and learning approach that is important at tertiary level.
Conclusion
The results reveal that DAT facilitate understanding of EST online materials, since DAT
facilitate the process of adding, editing and modifying information in electronic form without
making any changes to the resource itself. This was demonstrated in examples discerned fromthe annotations in which most students were able to identify paragraph structure and improveunderstanding of the reading materials. The various colors, such as yellow and green, in thehighlighting tools assist in reducing the cognitive overload because the highlighting tools
appear to provide meaningful cues for students. The highlighting process and written
annotations are significant in the process of contextualizing and synthesizing ideas, which

23
25

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

consequently leads to interactive reading. The process of contextualizing and synthesizing
ideas will enable students to be prepared and ready in dealing with syntactic and discoursefeatures that are generally used in the language of EST more easily. Furthermore, thehighlighting features allow students to focus on the reading task, while being engaged in the
annotation process. The annotations can also be seen as an additional layer of information to
the existing resources offline, such as discussion in class with the lecturer and other students.
To support this, data obtained on how the students identify the paragraph structure as part ofthe comprehension process in reading the materials were triangulated with annotation analyses.
The results confirmed the validity and importance of DAT.
With regards to the Schema Theory, where reading is considered a constructive and active
process that requires readers to understand information based on retrieving and applying
previous knowledge, highlighting texts significantly supports reading as it identifies key parts,
aids recall information, and acts as visual signals. Past studies have proven that learners’
comprehension would also be higher when important words, phrases and sentences arehighlighted as opposed to when they were not highlighted.
The present study is one of few studies that examined ESL students annotation types in reading
EST online materials in Malaysia. There is a need for future research identifying types ofannotations based on various ESP subjects such as English for Medical Purposes and English
for Legal Purposes. Different subject areas may affect learners use of annotations. Learners
may provide different forms of annotations given in different contexts. Different texts and tasks
may lead to different approaches in annotating by the learners. This study found a significancein online reading platforms such as iREAD in assisting students in reading EST online
materials in their SL. Overall, DAT was able to assist, facilitate and support learning; in a vitalarea. Reading is an important skill and plays a significant role in successful learning.
Acknowledgement
This research was supported by Universiti Malaysia Pahang (UMP) research ID: RDU
1603142.

24
26

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Abuseileek, A. F. (2012). The effect of computer-assisted cooperative learning methods andgroup size on EFL learners' achievement in communication skills. Computers &Education, 58, 231–239. https://doi.org/10.1016/j.compedu.2011.07.011
Ahern, T. C. (2005). Using online annotation software to provide timely feedback in an
introductory programming course. Proceedings of 35th ASEE/ IEEE frontiers ineducation conference. IEEE. https://doi.org/10.1109/FIE.2005.1611917
Anderson, N. J. (2003). Scrolling, clicking, and reading English: online reading strategies in asecond/foreign Language. Reading Matrix: An International Online Journal, 3(3), 1–
33.
Carrell, P. L., Devine, J., & Eskey, D. E. (1988). Interactive approaches to second language
reading. Cambridge Unversity Press. https://doi.org/10.1017/CBO9781139524513
Chen, Y.-C., Hwang, R.-H., & Wang, C.-Y. (2012). Development and evaluation of a Web
2.0 annotation system as a learning tool in an e-learning environment. Computers &Education, 58(4), 1094–1105. https://doi.org/10.1016/j.compedu.2011.12.017
Chiu-Jung, C., & Pei-Lin, L. (2012). Comparisons of learner-generated versus instructorprovided multimedia annotations. Turkish Online Journal of Educational Technology,
11(4), 72–83.
Cho, B.-Y., & Afflerbach, P. (2017). An evolving perspective of constructively responsive
reading comprehension strategies in multilayered digital text environments. In E. I.
Susan (Ed.), Handbook of research on reading comprehension. Guilford Press.
Chun, D. M., & Payne, J. S. (2004). What makes students click: Working memory and lookup behavior. System, 32(4), 481–503. https://doi.org/10.1016/j.system.2004.09.008
Creswell, J. W. (2014). Research Design : Qualitative, quantitative, and mixed methods
approaches. SAGE Publications.
Creswell, J. W. & Plano Clark, V. L. (2011) (Eds.). Designing and conducting mixed methods
research. SAGE Publications.
Duke, N. K., & Pearson, P. (2002). Effective practices for developing reading
comprehension. In A. E. Farstrup & S. Samuels (Eds.), What research has to say
about reading instruction. International Reading Association.
Gilbert, J. (2017). A study of ESL students’ perceptions of their digital reading. The Reading
Matrix: An International Online Journal, 17(2), 179–195.
Güzer, B., & Caner, H. (2014). The past, present and future of blended learning: An in depth
analysis of literature. Procedia - Social and Behavioral Sciences, 116, 4596–4603.
https://doi.org/10.1016/j.sbspro.2014.01.992
Haryani, H., Wan Faezah, A., & Nor Aini, A. R. (2012). The adoption of blended learning
among Malaysian academicians. Procedia - Social and Behavioral Sciences, 67, 175–
181. https://doi.org/10.1016/j.sbspro.2012.11.318
Herold, B. (2014). Q&A: Judging online vs. offline reading skills. Editorial Projects in
Education Inc.
Huang, H. (2014). Online versus paper-based instruction: comparing two strategy training
modules for improving reading comprehension. RELC Journal, 45 (2), 165–180.
https://doi.org/10.1177/0033688214534797

25
27

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Kiili, C. & Leu, D. J. (2019) Exploring the collaborative synthesis of information during
online reading. Computers in Human Behaviour, 95, 146–157.
https://doi.org/10/1016/j.chb.2019.01.033
Johnson, T. E., Archibald, T. N., & Tenenbaum, G. (2010). Individual and team annotation
effects on students’ reading comprehension, critical thinking, and meta-cognitive
skills. Computers in Human Behavior, 26(6), 1496–1507.
https://doi.org/10.1016/j.chb.2010.05.014
Leu, D. J., Zawilinski, L., Castek, J., Banerjee, M., Housand, B. C., Liu, Y., & O’Neil, M.
(2007). What is new about the new literacies of online reading comprehension? In A.
J. E. Leslie Susan Rush, Allen Berger, (Eds.), Secondary school literacy: What
research reveals for classroom practice (pp. 1–18). National Council of Teachers ofEnglish.
Li, L.-Y., Tseng, S.-T., & Chen, G.-D. (2016). Effects of hypertext highlighting on browsing,reading, and navigational performance. Computers in Human Behavior, 54, 318–325.
https://doi.org/ 10.1016/j.chb.2015.08.012
Lo, J.-J., Yeh, S.-W., & Sung, C.-S. (2013). Learning paragraph structure with online
annotations: An interactive approach to enhancing EFL reading comprehension.
System, 41, 413–427. https://doi.org/10.1016/j.system.2013.03.003
Marshall, C. C. (1997). Annotation: From paper books to the digital library. Proceedings ofthe 2nd ACM international conference on digital libraries. ACM.
https://doi.org/10.1145/263690.263806
Meyer, B. J. F., Wijekumar, K. Middlemiss, W., Higley, K., Lei, P. W., Meier, C., &Spielvogel, J. (2010). Web-based tutoring of the structure strategy with or withoutelaborated feedback or choice for fifth- and sevent-grade readers. Reading Research
Quarterley, 45(1), 62–92. https://doi.org/10.1598/RRQ.45.1.4
Moss, B. (2004). Teaching expository text structures through information trade book
retellings. Reading Teach, 57(8), 710–718. https://doi.org/ 10.2307/20205422
Nokelainen, P., Kurhila, J., Miettinen, M., Floreen, P., & Tirri, H. (2003). Evaluating the roleof a shared document-based annotation tool in learner-centered collaborative learning.
Proceedings of the 3rd IEEE international conference on advanced learningtechnologies. IEEE. https://doi.org/ 10.1109/ICALT.2003.1215056
Nor Fariza, M. N., Afendi, H., Hazita, A., Noorizah, M. N., & Vengadasamy, R. (2014).
Application of collaborative learning theory as a learning feature in iREAD UKM: A
conceptual Framework. In Pixel (Ed.), Proceedings of the ICT for language learning
conference. Librariauniversitaria.it Edizioni, (pp. 490–493).
Norazah, M. N., Mohamed Amin, E., & Zaidan, A. B. (2011). Summary of findings on thestatus, trends, & challenges of e-learning implementation. In E. Mohamad Amin(Ed.), E-learning in Malaysian higher education institutions: Status, trend andchallenges (pp. 107–115). Ministry of Higher Education.
Pardo, L. S. (2004). What every teacher needs to know about comprehension. Reading Teach,58 (3), 272–280. https://doi.org/10.1598/RT.58.3.5
Ruhil Amal, A., Nor Fariza, M. N., & Afendi, H. (2018). Using iREAD in understanding
online reading strategies applied by Science and Technology students. International
Journal of Web-based Learning and Teaching Technologies, 13(3), 18–32.
10.4018/IJWLTT.2018070102

26
28

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Rumelhart, D. E. (1977). Toward an interactive model of reading. In S. Dornic (Eds.),
Attention and performance VI (pp. 721–747). Erlbaum Associates.
Su, A. Y. S., Yang, S. J. H., Hwang, W.-Y., & Zhang, J. (2010). A Web 2.0-based
collaborative annotation system for enhancing knowledge sharing in collaborative
learning environments. Computers & Education, 55(2), 752–766.
https://doi.org/10.1016/j.compedu.2010.03.008
Sung, Y.-T., Wu, M.-D., Chen, C.-K., & Chang, K.-E. (2015). Examining the online reading
behavior and performance of fifth-graders: Evidence from eye-movement data.
Frontiers in Psychology, 6, 1–15. https://doi.org/10.3389/fpsyg.2015.00665
Thang, S. M., Wong, F. F., Noorizah, M. N., Rosniah, M., Najihah, M., & Kemboja, I.
(2012). Using a blended approach to teach English for academic purposes: Malaysian
students' perceptions of redesigned course materials. International Journal of
Pedagogies and Learning, 7(2), 142–153. https://doi.org/10.5172/ijpl.2012.7.2.142
Thoms, J. J., & Poole, F. (2017). Investigating linguistic, literary, and social affordances of
L2 collaborative reading. Language Learning and Technology, 21(2), 139–156.
https://doi.org/ 10125/44615
Tseng, S.-S., Yeh, H.-C., & Yang, S.-h. (2015). Promoting different reading comprehension
levels through online annotations. Computer Assisted Language Learning, 28(1), 41–
57. https://doi.org/10.1080/09588221.2014.927366
Wang, Q., Wooa, H. L., & Zhao, J. (2009). Investigating critical thinking and knowledge
construction in an interactive learning environment. Interactive Learning
Environments, 17(1), 95–104. https://doi.org/10.1080/10494820701706320
Corresponding author: Ruhil Amal Azmuddin
Contact email: ruhilamal@ump.edu.my

27
29

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

28
30

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Democratization of Education through Massive Open Online Courses in Asia
Runchana Pam Barger
Chiang Mai University, Thailand
and Wheaton College, USA

29
31

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
Massive Open Online Courses have been a recent phenomenon in providing large-scaleinteractive participation and open access to courses online. Depending upon internet
availability and familiarity with digital learning practices, this alternative could provide
education for many people. This paper explores whether technology such as massive online
open courses can democratize education by providing opportunities and access for those who
desire formal learning. This paper reviews literature on massive online open courses as well as
the benefits and challenges of these courses in adult education. Using computer-based content
analysis, this paper also examines recent research (2010-2019) on massive online open courses
and the implications of using them to provide widespread access to higher education in Asia.
The findings show that offerings in many Asian countries help promote social and economic
mobility for their people by providing continuing educational, professional, and personal
development through courses online. Yet barriers involving digital literacy, technicalcapabilities, and language as well as culture, prevent the underserved from pursuing this digital
education. This paper provides future research suggestions for collaboration of educationalorganizations to use massive online open courses in engaging life-long skills for people in Asia.
Keywords: MOOC, democratization, education access, technology, Asia

30
32

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

The theme of the 2015 World Education Forum, sponsored by the United Nations Educational,
Scientific, and Cultural Organization (UNESCO) and the Republic of Korea, was “equitable
and inclusive quality education and lifelong learning for all by 2030: transforming lives
through education” (UNESCO, 2015b, p. 11). The preamble of the Incheon Declaration definededucation as an essential and universal right for all humans to create and ensure peace,equipping people for employment with the hope that they have social and economic means for
self-sustainability (UNESCO, 2015a, Preamble 5). The report stressed the value of educationin promoting gender equality and equity by supporting gender-sensitive policies, improving
teacher training, and creating curriculum to decrease discrimination and violence against
females in schools (UNESCO, 2015a, Preamble 8). The preamble called for donors from
affluent nations to provide funding and resources, such as educational technology, to help
educators from all around the world provide quality of learning opportunities for women andthe underserved (UNESCO, 2015a, Preamble 9–16).
Based on the World Education Forum and the Incheon Declaration’s mandates, this paper
examines a web-based educational offering known as Massive Open Online Courses (MOOCs)
to explore as a potential means to increase educational affordability. MOOCs have provided
large-scale interactive participation and open access to courses online. Depending on theinternet availability in some of these developing countries, this alternative may provide adultsa chance to a obtain higher education degree. This paper explores whether technology such as
MOOCs can democratize education by providing opportunities and access for those who desire
higher education and professional development. This paper also reviews literature on MOOCs
as well as the benefits and challenges of them in adult education. Specifically, the paper
examines recent research (2010-2019) related to the massive open online courses in Asia andapplies content analysis to explore themes and associations with technology and its
implications in providing education for people in Asia, including the underserved communities.
Literature Review
Since the 1990s, online learning has been on the rise (Alcorn et al., 2015). A recent report fromthe Babson Survey Research Group reports that distance education has grown dramaticallyover the past fourteen years (Seaman et al., 2018). In particular, public research universities
have increasingly offered MOOCs as a tool for students and senior academic leaders believethat MOOCs will attract potential students (Allen & Seaman, 2013).
Many claim to have coined the term MOOC. Some researchers have stated that it originatedduring the early 2000s with the emergence of technologies such as open source and open
courseware platforms (Bozkurt et al., 2016; Zawacki-Richter & Naidu, 2016). Some sources
attribute the term MOOC to Daniel Barwick, an associate professor of philosophy in New
York. In 2007, Barwick discussed the need for major universities to find methods to improvelearning for large numbers of students (Lederman, 2007; Seaman et al., 2018). Other sources
say that Dave Cormier, the Manager of Web Communication and Innovations at the Universityof Prince Edward Island, coined the term MOOC (Bozkurt et. al, 2016; Hollands & Tirthali,2014). Cormier pointed out that over 2,000 students from the general public took a 2008
University of Manitoba online course, titled “Connectivism and Connective Knowledge,” that
George Seimens and Stephen Downs created, at no cost (Adham & Lundquist, 2015; Alcorn
et al., 2015; Bozkurt et al., 2016; Hollands & Tirthali, 2014). The term MOOCs gained evengreater popularity after an artificial intelligence online class at Stanford University logged
enrollment of over 150,000 students (Alcorn et al., 2015; Waldrop, 2013).

31
33

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

MOOCs have three overall characteristics: they are courses that are massive, open, and online.
They are massive in that they are characterized by a very large number of enrolled students
from the range of hundreds to hundreds of thousands. They are also massive in that they provide
large-scale interactive participation for the public. MOOCs are open in that they often use
learning platforms that are open-source and they are free or low-cost to anyone who isinterested. They are online in that curriculum and assessment are also open so anyone who has
an internet connection can join the course. An important attribute of these digital classes is thatlearners are free to study anywhere without restrictions or limitations (Adham & Lundquist,2015).
MOOCs are categorized as either “cMOOCs” or “xMOOCs” based on their differing
philosophies and methodologies. Connectivist MOOCs, also known as cMOOCs, are courses
in which learners use learning platforms such as wikis, social media, blogs, or websites such
as Peer to Peer University (www.p2pu.org) to network and collaborate with one another (Bates,
2014; Smyrnova-Trybulska, et al., 2016). CMOOCs are heavily dependent upon learning
communities to create knowledge together (Adham & Lundquist, 2015; Lane, 2016). Important
tenets of cMOOCs include autonomy of the learner, diverse demographics of learners,
interaction and cooperative learning between participants, and openness in the courses (Bates,2014). XMOOCs depend upon traditional classroom structures containing specificallydesigned platform software, video lectures, and automated assessments (Adham & Lundquist,
2015; Bates, 2014; Smyrnova-Trybulksa et al., 2016). Coursera, Udacity, and edX courses areexamples of xMOOCs. XMOOCs provide alternatives to the traditional university residential
model, as students take courses online. XMOOCs focus on a teaching-centered model rather
than the learner-centered focus of cMOOCs. Because of the more formal nature of xMOOCs,
students who take xMOOCs can earn certificates and they are more popular than cMOOCs
because of their instructivist content. Recent online education research examines xMOOCs,
and therefore this particular paper focuses on the latter course offerings.
Democratization as Comparative Education Framework
Comparative education is the application of social scientific methods and theories tointernational issues of education (Epstein, 2002). One issue that comparativists have examinedis the role of democratization on education. An underlying notion of democracy is the humanright to vote in a political process, to have accessible distribution of financial resources, and tohave the essential right to be treated justly (Davis, 2008). Comparative education examines theimpact of democratization in local, national, and global communities (McGinn, 1996). In an
ideally democratized community, people would be able to freely make their own decisions thatwould benefit themselves locally and globally. This includes decisions involving education
access and opportunities for their citizens. This section examines the political, educational, andsocial impacts that democratization has on international education.
Comparative education research examines the role of democratization and politicalparticipation (Blankenship & Kubicek, 2018; McGinn, 1996; Murati, 2015; Zajda, 2008).
Research examines the correlation of active participation in politics when governments
promote education access and opportunities (McGinn, 1996; Murati, 2015). Democratization
through citizenship education promotes peace and freedom, and schools should be institutions
that allow them to work on democratic principles to influence society (Murati, 2015).
Comparative education research shows mixed results on the impact of democratization in
emerging countries. Countries that attempt to promote democratization have not implemented
it well typically have political corruption such as the lack of accountability of public officials

32
34

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

as well as mismanagement of governmental funding (Powell et al., 2016). In Pakistan, forexample, educational disparity is widened as the government has not provided adequate
funding for education in the countryside versus their urban counterparts (Powell et al., 2016).
However, other countries, such as Ukraine, attempt to democratize education throughimproving technology. One way in which Ukraine’s educational system has reduced academic
corruption is through data management software to protect privacy, systematize registration,and track grades for their students (Powell et al., 2016).
Comparativists have also researched democratization and its educational impacts in
implementing free markets and capitalist ethics (Davies, 2008). Economically, a democratized
government would distribute funding for education, incorporate civic education, anddecentralize power to local officials (Davies, 2008; McGinn, 1996; Murati, 2015).
Democratization in education should include decision-making from all aspects of an
educational institution from the faculty, staff, parents, and students (McGinn, 1996; Murati,2015). A democratic education curriculum incorporates teaching and learning of skills such as
direct communication, deliberative process, negotiation, collaborative sharing, critical thinking
and problem solving (McGinn, 1996).
Furthermore, democratization in education would also promote equitable education and accessto educational opportunities to benefit all social groups (OECD, 2006; Zajda, 2008).
Educational equity includes teaching and learning social justice issues, raising consciousnessof power relations and providing materials to share voices of all groups of people, especiallythe marginalized (Sefa Dei & Shahjahan, 2008). Sefa Dei and Shahjahan, research educatorsin the field of equity and diversity studies, stated that “equity affirms that physical, material,
emotional, social, and spiritual well-being of both self and collective… the affirmation arises
from the recognition of each other’s fundamental freedoms and rights to valued goods andservices of society, while at the same time fulfilling accompanying responsibilities” (2008, p.
49). Comparative education research also examines the relationship between democratization
and women’s rights. These include an increase in participation of women in politics, written
policies and laws advancing gender equality, a reduction in gender-based violence, funds for
women’s health, and alterations in family law to support women (Blankenship & Kubicek,2018).
Methodology
This paper examines the relationship between online education and democratization of
education. In particular, the main research question is as follows: Does online education,
specifically using MOOCs, provide for the democratization of education for people in Asia.
Democratization is examined in terms of allowing people to have the choice to not only learn,
but also to participate in their society, whether it is socially, politically, or economically. This
research explores whether people, especially the underserved, can access online education thatis effective for their professional and/or personal growth.
This study explores research on the efficacy of massive open online courses to democratization,
which was carried out from 2010 to 2019. The research method used is content analysis, which
examines the frequency of terms, such as words and phrases, in texts to make inferences about
the associations and possible connotations within the documents (Columbia University
Mailman School of Public Health, 2019). Content analysis can be both quantitative andqualitative (Seuring & Gold, 2011). However, this study uses qualitative methods as documents
and texts are examined. Types of content analysis include conceptual and relational analysis.

33
35

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Conceptual analysis examines patterns or themes in texts that occur often, whereas relational
analysis explores the relationship between concepts and its importance (Columbia University
Mailman School of Public Health, 2019). The study explored the conceptual and relational
links between massive open online courses, educational access, and opportunities for people.
Initially, the process began as a deductive approach examining whether MOOCs democratize
education for people in various countries in Asia. In particular, the initial research question
focused on whether some Asian educational ministries or institutions offer low-cost coursesthat would benefit academic as well as vocational skills to their people. Specific themes suchas the history, policies, benefits, challenges, and suggestions emerged in MOOC research on
education access and opportunities through manual document analysis. An inductive approach
was used to understand the more recent concepts or themes written in the literature from 2015
to the present.
The inductive approach used for this study employed a qualitative content analysis collectedfrom 48 academic books and journals on issues related to MOOCS in Asia. The author hoped
to identify MOOC trends relating to education access and opportunities. In particular, theresearch employed a similar type of methodology to Zawacki-Richter et al.’s (2018) automatedcontent analysis in identifying themes and concepts of MOOC research using the program
Leximancer. This text-mining statistical tool carefully extracts themes from documents tolocate patterns and trends. Leximancer compiles various texts and documents, codes common
themes, and creates a visual concept map creating categories that connect frequently-occurring
information from the documents (Leximancer, n.d.). Figure 1 shows the phases of the research
process that I conducted:

Figure 1: Phases of the Research Process
Forty-eight academic journals and book chapters were selected using the keywords MOOCs,
MOOC, and Asia. These were found in two research databases: Google Scholar and ERIC. Allof the 48 documents were selected using the following criteria: peer-reviewed journal or bookchapter specifically focusing on MOOCs in Asia from 2010-2019, written in English, andonline full-text accessibility. Limitations of the study include a smaller content analysis sample(48 documents) and the documents selected for analysis only being written in English, whichnarrowed the range of research able to be analyzed.
In addition to the manual coding of certain democratization themes such as access, opportunity,
provision, literacy, mobility, Leximancer breaks down the themes in the 48 documents tocategories and relationships between those categories to create a visual concept map.

34
36

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Results and Discussion
Figure 2 illustrates the concept map derived from Leximancer, the automated content analysis
software. The concept map contains concept circles of frequent words or phrases that areconnected to other concepts on the map (Leximancer Pty Ltd, 2018).

Figure 2: Concept Map of MOOCs in Asia
The concept map generates the highest-level occurring terms. Important terms are heat-colored;
for example, the “hottest” important concept appears in red, the next hottest is orange, andcontinues according to the color wheel (Leximancer Pty Ltd, 2018). In Figure 2, the hottest
concepts appear to be MOOCs and learning, while education and use are the next mostsignificant terms. Figure 3 below shows the interconnectivity of the concepts. Concepts thatappear together often are in the same pieces of documents (Leximancer Pty Ltd, 2018).

Figure 3: Relationships between Concepts

35
37

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

From Figure 3, the learning and MOOC red circles appear to be very connected with otherconcept circles as seen with the dark red lines. The learning and MOOC circles, which are thehottest or most-frequented concepts connect to other concept terms of development, provide,courses, education, universities and use. This concept summary indicates the relative
importance of education, learning, and MOOCs as compared to other terms. From this analysis,
three themes have emerged because of the text-mining analysis and the concept map that relates
to democratization of education: miscellany of MOOCs in Asia; provision and development of
MOOC education in Asia; and opportunities and barriers of Asian learners to access MOOCs.
Miscellany of MOOCs in Asia
From the content analysis, the MOOC concept circle is the central theme of the literature. The
sub-themes within this concept circle include terms such as online, courses, open, platform,
and massive. The text-mining analysis connected the importance of MOOCs in Asia and thevariety of MOOC offerings provided in countries such as Malaysia, India, Nepal, and China.
Some Asian universities have used MOOC platforms such as Coursera, edX, and Udacity,
created by the collaboration of U.S. top universities and companies (Liyanagunawardena et al.,2013; Chen, 2013; Beigi et al., 2015). Countries such as India and Brazil contribute 14 percent
of enrolled students in edX, Coursera, and Udacity (Kanwar, 2014). The National Universityof Singapore joined Coursera, while Nanyang Technology joined ITunes U (Chen, 2013). In
2012, edX announced its growing network to many international universities, including Asianuniversities, to meet global demand and increase access to quality education (edX, 2013). Asianuniversities include Tsinghua University and Peking University in China, the University ofHong Kong and Hong Kong University of Science and Technology in Hong Kong, Kyoto
University in Japan, and Seoul National University in South Korea (edX, 2013).
As global edX, Coursera, and Udacity platforms have their online courses in English, many
countries in Asia have adapted to create MOOCs for their own cultural and linguistic context(Buhl & Andreason, 2018). Some nation-states have prioritized MOOCs as part of theirnational education plans (Buhl & Andreason, 2018). Countries that have created local MOOCs
as part of their educational strategies include Taiwan, China, Indonesia, Japan, Malaysia,
Philippines, South Korea, Singapore and Thailand (Chen, 2013; Laurillard & Kennedy, 2017;
Sari et al., 2019). In 2013, Tsinghua University founded the XuetangX platform in China,
which also is part of the Chinese Association of Higher Education (Laurillard & Kennedy;
Xuetangx, 2019). In Indonesia and Malaysia, MOOCs have been derived from government
initiatives to create access to education for their people, especially when brick and mortar
institutions are limited (Norman et al., 2015; Sari et al, 2019). In 2015, the Malaysian Ministryof Education collaborated with four public universities, National University of Malaysia,
University Putra Malaysia, MARA University of Technology, and University of Malaysia
Sarawak, to launch Malaysia MOOCs (Nordin et al., 2016). The Ministry of Education andCulture, along with Ciputra University and Universita Terbuka, collaborated in creating
MOOCs in Indonesia (Sari et al., 2019). In Japan and Korea, JMOOCs and KMOOCs werecreated by an inter-organizational conglomerate of academic, business, and governmental
institutions (Shigeta et al., 2017; Upadhyay & Anandpara, 2020,). Similar to the East Asian
counterparts, ThaiMOOCs, launched by universities and governmental initiatives in Thailand,
had their aim to develop life-long opportunities, innovation, and quality education for diverse
groups of people (Chaimin, 2019). SWAYAM, otherwise known as Study Webs of ActiveLearning for Young Aspiring Minds, “is a progamme initiated by the Government of India anddesigned to achieve the three cardinal principles of education policy including access, equity
and quality…to take the best teaching learning resources to all, including the most

36
38

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

disadvantaged” (SWAYAM, 2019, para. 1). Furthermore, Katmandu University in Nepal has
recently piloted a MOOC course on basic programming which is one of the first locallydeveloped MOOCs in the country (Ghimire, 2018; Singh, 2018).
Educational Development and Provision of MOOCs
In the concept map of MOOC academic literature from 2010-2019, the education term
emerged. The content analysis of the research literature describes the MOOC impact onproviding education opportunities and access to people. In implementing the analysis andinterpretation of each the 48 academic journals and book chapters in detail (see Figure 1: Step4), another theme that emerged is the educational development and provision of MOOCs.
Factors that can hinder democratization of education of MOOCs to those people are thephysical infrastructure of technical capabilities and digital literacies of people understanding
how to use MOOCs.
In India, where there could be possibly 40 million extra university student spots needed by thenext decade, MOOCs in SWAYAM can help learners to achieve as much as 20 percent of theireducational curriculum (Buhl & Andreasen, 2018; Chatterjee & Nath, 2014). MOOC advocatesin India hope that online learning can help increase the literacy rates and help learners with
employment training (Upadhyay & Anandpara, 2020). The rise of Indonesian MOOCs, offeredat higher education institutions such as Ciputra University and Universita Terbuka, aim to help
the underserved to find quality education at an affordable and accessible cost (Sari et al., 2019).
MOOCs can also offer benefits to Nepali society such as open and free learning opportunities,
affordable and inclusive education, and developing Nepalese teacher professional development(Ghimire, 2018; Singh, 2018).
Technical capabilities. According to Liyanagunawardena, Williams, and Adams (2013),
access to educational technology encompasses the physical, motivational, and intellectual
knowledge of understanding how to use digital devices. Some studies find a disparity for lessprivileged groups in obtaining internet access in various parts of the world. For some countries
in Asia, internet access can be a challenge because of poor national infrastructure. For example,
parts of Sri Lanka have limited electricity and their internet centers tend to be in more urban
areas (Liyanagunawardena et al., 2013). As recently as in 2017, internet usage in Nepal wasless than 50 percent (Ghimire, 2017). Furthermore, Southeast Asian learners cannot afford
electronic equipment to enroll in MOOCs (van de Oudeweetering & Agirdag, 2015).
Access for women is more restricted where the internet is sparse. Women in Asia often go to
public libraries rather than internet cafes because they perceive the former as safer (Laurillard& Kennedy, 2017). According to the World Bank, women also have less access to certain
technology devices, as over 1.7 billion women in low- and middle-income countries do not
own mobile phones (World Bank, 2016). Furthermore, women have less access due to cultural
norms and pressures from families and friends not to use the internet (Laurillard & Kennedy,2017).
Slow internet connection poses an obstacle for all learners accessing MOOCs that contain large
files or that stream videos (Liyanagunawardena et al., 2013). In a survey conducted with
Nepalese engineering students who took the MOOC course entitled “Fundamental Concepts of
C Programming”, the students stated that slow internet access was a main barrier to MOOC
participation and completion (Shakya & Shrestha, 2018). Poor internet connection, especiallyin small towns and rural regions, can disrupt learning in various parts of the world (Adham &Lundquist, 2015). However, some governments and other organizations have developed

37
39

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

MOOCs to support low-bandwidth connections. Murugesan, Nobes, and Wild (2017) observeda MOOC course for researchers from developing countries. This research and writing course
was delivered in a Moodle site of low-bandwidth methods including high-quality textual
content and videos or graphics that were low in size. Over 2,830 learners from over 90 countries
participated in the course. In this low-bandwidth methods course, they found that over half ofthe learners were from the Majority World including Asian countries such as Sri Lanka, India,
Nepal, and the Philippines (Murugesan et al., 2017). Over 45 percent of the participants in the
MOOC research course were women. Furthermore, contrary to the typical findings, the women
had a higher completion rate in relation to the men, engaging better in discussion forums andreflective dialogue, which is an indicator of digital literacy as well (Murugesan et al., 2017).
Digital literacy. Access to technology does not equate with proficiency of use. A young Sri
Lankan female teacher once responded in an interview about using MOOCs that although she
had good internet connectivity and technological resources, she did not know how to use them
(Liyanagunawardena et al., 2013). Her response underscores that for women and theunderserved in Asia, digital literacy is vital to the sustainability and completion of MOOCs.
Digital literacy includes information processing, competently navigating online tools, building
community through discussion forums, self-discipline in reading and completing assignments,
tasks, and quizzes, and having the ability to self-learn (Trehan et al., 2017). For many nationsin the world, computer literacy is still in the early stages. A little over 20 percent of Sri Lankan
adults are proficient at using digital technology (Liyanagunawardena et al., 2013). In Chineseand Indian universities, those with information literacy, that is, better internet skills and thedesire to learn through videos, have been more likely to succeed in MOOCs than those who
were not as technically able (Trehan et al., 2017). Even educators and students who have been
exposed to technology have a difficult time in understanding the process of implementing or
using MOOCs. In India, teaching assistants who helped the lead instructors for the SWAYAM
online courses have a challenging time with technical skills and handling of MOOCs (Buhl &Andreasen, 2018). For Nepalese students who took the pilot MOOC course in Kathmandu
University, they expressed confusion regarding the assignment expectations such as posting
discussions that they eventually dropped out of the course (Ghimire, 2018). To counteract theseissues with digital literacy, recommendations for improvement in MOOC retention include
clear instructions with modeling examples from the instructor and technology orientation(Ghimire, 2018). This is because prerequisite knowledge of using MOOCs, such as backgroundknowledge and programming skills, poses additional barriers for some women and theunderserved in Asia in taking MOOCs (van de Oudeweetering & Agirdag, 2015).
Lingual/Cultural/Social/Economical Factors for Asian Learners of MOOCs
Another category, which emerged in the concept map analyzing MOOCs in Asia documentsfrom 2010-2019, is learning. The learning concept examines the social, cultural, and physicalaspects of the learners and it appears that the design and pedagogy of MOOCs influences them.
Cultural differences of Asian online learners can be a factor in MOOC education, as well as
language, communication tool use, plagiarism, time zone differences, instructional styles,
language and other cultural factors (Chen, 2013).
Language and culture. Not only do women learners and the underserved in Asia need to havesufficient digital literacy, they need to understand English. Many MOOCs originated in
America and Europe where their universities and providers created the learning platforms(Laurillard & Kennedy, 2017). Only a small proportion of students in the Majority World are
multilingual (Liyanagunawardena et al., 2013). This creates obvious challenges, as seen in
MOOC creation in Nepal. This includes difficulty in the comprehension of the video lectures

38
40

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

and texts, the pace and tone of the lecturers, creating discussion posts, and understanding
assessment as English is not their first language (Ghimire, 2020).
Culture also plays a role in completing online courses. MOOCs have certain discourses suchas instructions, humor, or content in the discussion forums that other diverse learners may notunderstand (Liyanagunawardena et al., 2013). In some cultures, quality learner engagement
involves direct teacher to student interaction (Laurillard & Kennedy, 2017). Interculturalcommunication also is a factor as students from collective and high context cultures, like thosein Asia, stated problems in understanding their classmates’ viewpoints without visual signals(Bayeck et al., 2018). In a required Malaysian Ethnic Relations MOOC undergraduate course,
students expressed that having Malaysian instructors who speak the same language as them isa factor in helping them understand the video lectures better as the teachers can identify with
their local culture (Nordin et al., 2016). In many countries in Asia such as Nepal and India,
traditional face-to-face classroom, rote learning and plagiarism is engrained in the education
context; thus, instructors of MOOCs need to be aware of the challenges they faced whenassessing their students (Ghimire 2020; Mohapatra & Mohanty, 2017).
Suggestions to improve the quality of MOOCs to their diverse learners include providing
inclusive guidelines to respectively post on discussion forums, a variety of audio-visualmaterial, and flexibility on assessments to attract those with different learning styles(Liyanagunawardena et al., 2013; Laurillard & Kennedy, 2017). Furthermore, learning
engagement would increase if MOOCs enabled their diverse learners to share their personal
stories with one another, using local and global contexts to help them understand the course
content.
Regarding gender, MOOCs have the potential to offer and expand educational opportunities
for women in Asia and the Majority World (Alcorn et al., 2015; Liyanagunawardena et al.,2013). Liyanagunawardena et al., (2013) state that MOOCs can benefit females in nations
where access to education would be limited, such as in Afghanistan or in Nepal. Video
conferencing on the social value of MOOC based discussions can provide women access tomore senior global faculty than they would have in their local classroom (Laurillard &Kennedy, 2017).
MOOCs have the potential to democratize education for people with limited learning
opportunities, as universities, commercial, government, and non-profit providers realize thatthese open online courses cannot be a “one-size-fits-all” platform. Creators of MOOCs need totake into account the context of the learners, including the national infrastructure, schooling,technical educational background, language, and culture.
Economic and social mobility. As countries in Asia seek more labor, MOOCs can be a viableoption for women and those who cannot afford traditional higher education to improve jobrelated skills. A large number of Indian learners have signed up for edX MOOCs at Harvard,
MIT, and Penn (Alcorn et al., 2015). About 20 percent of the students in these MOOC courses
are women and those in rural residences of India (Alcorn et. al, 2015). Many Indian learners
have used MOOCs for professional development, to improve their current job skills, or totransition to new positions, primarily in STEM fields. In a 2014 survey of 780,000 learners
who completed a Coursera MOOC course from approximately 212 countries, 72 percent of thesurvey respondents chose career benefits and 61 percent reported educational benefits(Zhenghao et al., 2015). Interestingly, people from Asia, people with lower socioeconomic
status, and those with less education reported they benefited more from MOOCs than their elite

39
41

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

counterparts (van de Oudeweetering & Agirdag, 2015; Zhenghao et al., 2015). The former
respondents stated that completing a Coursera MOOC assisted them with tangible career
benefits such as getting a raise, finding a new job, or improving upon a skill for a job promotion(Zhenghao et al., 2015). Furthermore, the survey responders from the Majority World such as
those in Asia and people with lower socioeconomic status said they believed that the MOOCs
helped them improve their educational output, (i.e., transferring the MOOC to academic credit
or attaining knowledge). Other studies support the trends that when learning job-specific skills,
learners from the Majority World such as those in the Philippines were more likely to complete
MOOCs than those in the Western countries (Garrido et al., 2016; Murugesan et al., 2017).
MOOCs that cater towards women and the learners of the Majority World through their content
are likely to have their learners complete their courses. For example, in a MOOC entitled“Understanding Dementia”, learners can retake assessments as many times as necessary forthem to understand the content (van de Oudeweetering & Agirdag, 2015). Another MOOC
known as “Web Science” contains materials that non-native English learners can understand(van de Oudeweetering & Agirdag, 2015). In a MOOC on research writing for researchers indeveloping countries, the low-bandwidth materials in the course and the asynchronous
flexibility of the course allowed female learners to complete the course, especially as many ofthem have additional work and familial responsibilities (Murugesan et al., 2017).
Economic and social mobility are motivating factors for some women enrolling in MOOCs.
Other motivating factors include the participation of a friend or colleague in an online platform(Bayeck, 2016). A higher number of women took a Penn State Coursera MOOC course on“Creativity, Innovation, and Change”. The course attracted participants from over 82 different
nations, including China, the United States, India, Canada, Mexico, Brazil, Nigeria, Egypt,France, and Germany (Bayeck, 2016). Over 90 percent of the female respondents mentionedthat they took the course because of their friend. Eighty-one percent of respondents identified
job-related skills and connection with others as reasons that they took the course (Bayeck,2016). In terms of female participation, group work and collaboration were important factors
for women to complete the courses, especially as the topic of the course was engineering.
Conclusion and Future Research
MOOCs are open, low-cost, and unrestricted for any learner to enroll in them. Educators havepredicted that internet platforms would result in democratization of education by providing
educational access for all. They further have identified MOOCs as allowing those who havebeen excluded from residential higher education to participate in online courses (Adham andLundquist, 2015). In reviewing the literature and engaging in qualitative content analysis on
the impact of MOOCs on the democratization of education in Asia, researchers have examinedthe effects of technical capabilities, digital literacies, and language, as well as culture, on
women and the less privileged potentially using MOOCs for educational opportunities. Many
Asian universities have used a variety of MOOCs as part of their national plans to provide
opportunities for their people to receive higher education or continuing professional
development. Countries that have higher enrollments of people using MOOCs include thosethat have solid technical infrastructure and educational policies encouraging their people toenroll in MOOCs as part of their educational requirements. Recent research on MOOCs in Asia
still shows a disparity of enrollees who are either male, people in their 20s-30s, those who livein the urban areas, middle-to-upper social class, and learners who are competent in English.
Yet more countries in Asia, such as in Nepal, Indonesia, and Malaysia are providing more

40
42

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

internet availability and educational curriculum that provide digital literacy and MOOCs intheir local language to help the underserved with support to use MOOCs.
Multitudes of learners from around the world have enrolled in MOOCs. However, not alllearners have taken advantage of the online learning platforms. To make MOOCs even moreaccessible to females and the Majority World, MOOC providers in the United States andabroad should collaborate with various organizations such as governments, multinational
corporate sponsors, non-government organizations and community centers to work forunderserved groups to maximize the potential in providing education to these groups. The
democratization of education through MOOCs requires partnering organizations to improve
the MOOCs infrastructure, offer better broadband internet connectivity, and equip the lessprivileged groups with sufficient digital literacy to enable them to maximize the benefits
MOOCs have to offer. MOOC providers should also research and take the time to invest in
culturally and linguistically responsive content for their potential clients. As populations in the
Majority World increase, including those in Asia, the need to utilize technology will rise.
Traditional brick and mortar higher education institutions will not be able to accommodate
such multitudes of people. In order to meet the 2015 World Education Forum’s aim of equitable
and inclusive quality education and lifelong learning for all by 2030, local governments, grassroot organizations, and donors from affluent nations need to provide ample technological andeducational resources to create more MOOCs for women and those in the Majority World.
Further research will examine the extent to which global-based MOOCs such as Coursera or
edX, and local-based MOOCs in China, India, Indonesia, Japan, Malaysia, Nepal, Thailand,and other Asian nation-states have altruistic intentions of providing educational access, equity,and opportunity for women and the underserved in Asia.

41
43

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Adham, R. S. & Lundquist, K. O. (2015). MOOCs as a method of distance education in the
Arab world: A review paper. European Journal of Open, Distance and e-Learning,
18(1), 123–139. https://doi.org/10.1515/eurodl-2015-0009
Alcorn, B., Christensen, G., & Kapur, D. (2015). Higher education and MOOCs in India andthe Global South. Change: The Magazine of Higher Learning, 47(3), 42–49.
https://doi.org/10.1080/00091383.2015.1040710
Allen, I. E. & Seaman, J. (2013). Changing course: Ten years of tracking online education in
the United States. Babson Park, MA: Babson Survey Research Group and Quahog
Research Group. https://www.onlinelearningsurvey.com/reports/changingcourse.pdf
Bates, T. (2014, October 13). Comparing xMOOCs and cMOOCs: Philosophy and practice.
Online Learning and Distance Education Resources.
https://www.tonybates.ca/2014/10/13/comparing-xmoocs-and-cmoocs-philosophyand-practice/
Bayeck, R. Y. (2016). Exploratory study of MOOC learners’ demographics and motivation:
The case of students involved in groups. Open Praxis, 8(3), 223–333. https://doi.org/10.5944/openpraxis.8.3.282
Bayeck, R. Y., Hristova, A. Jablowkow, J. W., & Bonafini, F. (2018). Exploring therelevance of single-gender group formation: What we learn from a massive open
online course (MOOC). British Journal of Educational Psychology, 49(1), 88–100.
https://doi.org/10.1111/bjet.12528
Beigi, M., Wang, J. and Shirmohammadi, M. (2015). ARHD! Take the opportunity andpioneer vocational MOOCs. Human Resource Development International, 18(2),203–212. https://doi.org/10.1080/13678868.2014.978663
Blankenship, J. & Kubicek, P. (2018). Democratization and gender equality in Sub-SaharanAfrica. The Journal of the Middle East and Africa, 9(1), 27–50.
https://doi.org/10.1080/21520844.2018.1449458
Bozkurt, A., Keskin, N. O., & de Waard, I. (2016). Research trends in massive open online
course (MOOC) theses and dissertations: Surfing the tsunami wave. Open Praxis,
8(3), 203–221. https://doi.org/10.5944/openpraxis.8.3.287
Buhl, M. & Andreasen, L. B. (2018). Learning potentials and educational challenges ofmassive open online courses (MOOCs) in lifelong learning. International Review of
Education, 64, 151–160. https://doi.org/10.1007/s11159-018-9716-z
Chaimin, C. (2019). MOOC: Lifelong learning in the 21st century. Journal of Humanities andSocial Sciences, Chiang Mai Rajabhat University, 46–70.
https://so01.tci-thaijo.org/index.php/husocjr/article/download/201022/142538/
Chatterjee, P. & Nath, A. (2014). Massive open online courses (MOOCs) in higher education- Unleashing the potential in India. Proceedings of 2014 IEEE international
conference on MOOC, innovation, and technology in education, (pp. 256–260). IEEE.
https://doi.org/10.1109/MITE.2014.7020283
Chen, J. C. (2013). Opportunities and challenges of MOOCs: Perspectives in Asia.
IFLA WILIC 2013, Singapore. http://library.ifla.org/157/1/098-chen-en.pdf.

42
44

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Columbia University Mailman School of Public Health. (2019). Content Analysis.
https://www.mailman.columbia.edu/research/population-health-methods/contentanalysis
Davies, L. (2008). Interruptive democracy in education. In J. Zajda, L. Davies & S.
Majhanovich (Eds.), Comparative and global pedagogies: Equity, access, anddemocracy in education (pp. 15–32). Springer.
EdX (2013, May 21). EdX continues to grow its network of the world’s leading institutions ofhigher education to meet global demand and increase access to quality education[Press release]. https://www.edx.org/press/edx-expands-xconsortium-asia-doubles
Epstein, E. H. (2002). Globalization of education. In J. W. Guthrie (Ed.), Encyclopedia ofeducation (2nd ed., pp. 936–939). Macmillan.
Garrido, M., Koepke, L., Andersen, S., Mena, A. F., Macapagal, M., & Dalvit, L. (2016). An
examination of MOOC usage for professional workforce development outcomes inColombia, the Philippines, & South Africa. Seattle: Technology & Social ChangeGroup, University of Washington Information School.
https://tascha.uw.edu/publications/an-examination-of-mooc-usage-for-professionalworkforce-development-outcomes/
Ghimire, B. K. (2018). Maximizing continuous professional and academic development
through MOOC. In NELTA ELT Forum 2018 volume 4. Kathmandu: Nepal English
Language Teacher’s Association. (114–123).
Hollands, F. M., & Tirthali, D. (2014). MOOCs: Expectations and reality. Full report. Center
for Benefit-Cost Studies of Education, Teachers College, Columbia University, NY.
https://files.eric.ed.gov/fulltext/ED547237.pdf
Kanwar, A. (2014, March 28). Old wine in new bottles? Exploring MOOCs [White paper].
Commonwealth of Learning, 1–7.
http://dspace.col.org/bitstream/handle/11599/1019/2014_Kanwar_Old_Wine_Transcr
ipt.pdf?sequence=1&isAllowed=yLane, A. (2016). Emancipation through open education: Rhetoric or reality? In P. Blessinger
and T. J. Bliss (Eds.), Open Education: International Perspectives in Higher
Education (pp. 31–50). Open Book Publishers. https://doi.org/10.11647/OBP.0103.02
Laurillard, D. and Kennedy, E. (2017). The potential of MOOCs for learning at scale in the
Global South. Centre for Global Higher Education.
http://www.researchcghe.org/perch/resources/publications/wp31.pdf
Lederman, D. (2007, December 6). Does class size matter: Daniel Barwick challenges theassumption that small is always best in the college classroom. Inside Higher Ed.
https://www.insidehighered.com/views/2007/12/06/does-class-size-matter
Leximancer. (n.d.). What is Leximancer: Automatic Content Analysis [PowerPoint slides].
https://static1.squarespace.com/static/539bebd7e4b045b6dc97e4f7/t/5b0f8eab70a6ad
58c23e862c/1527746234314/Leximancer+Introduction.pdf
Leximancer Pty Ltd. (2018, April 12). Leximancer User Guide, Release 4.5.
https://doc.leximancer.com/doc/LeximancerManual.pdf

4345

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Liyanagunawardena, T. R., Williams, S., & Adams, A. A. (2013). The impact and reach of
MOOCs: A developing countries’ perspective. eLearning Papers, 38–46.
http://centaur.reading.ac.uk/32452/1/In-depth_33_1.pdf
McGinn, N. (1996). Education, democratization, and globalization: A challenge forcomparative education. Comparative Education Review, 40(4), 341–357.
https://doi.org/10.1086/447398
Mohapatra, S. & Mohanty, R. (2017). Adopting MOOCs for affordable quality education.
Education and Information Technologies, 22(5), 2027–2053.
https://doi.org/10.1007/s10639-016-9526-5
Murati, R. (2015). Conception and definition of the democratization of education. Journal of
Education and Practice, 6(30), 173–183.
https://files.eric.ed.gov/fulltext/EJ1081375.pdf
Murugesan, R., Nobes, A., & Wild, J. (2017). A MOOC approach for training researchers indeveloping countries. Open Praxis, 9(1), 45–57.
https://doi.org/10.5944/openpraxis.9.1.476
Nordin, N., Norman, H., Embi, M. A., Mansor, A. Z., Idris, F. (2016). Factors fordevelopment of learning content and task for MOOCs in an Asian Context.
International Education Studies, 9(5), 48–59. https://doi.org/10.5539/ies.v9n5p48
Norman, H., Dogan, H., Nordin, N., Mahamod, Z., Halim, L. (2015). An Asia-Europe
massive open online course for lecturer training development of pedagogicalstrategies for MOOCs. Proceedings of Public University Dean’s Council 2015
Seminar, Malaysia. https://www.researchgate.net/publication/281782494_An_AsiaEurope_Massive_Open_Online_Course_for_Lecturer_Training_Development_of_Pe
dagogical_Strategies_for_MOOCs
OECD. (2006). Education Policy Analysis: Focus on Higher Education. Paris. OECD.
Powell, D.V., Kuzmina, S., Cwick, S., Khalid, T. & Sofo, S. (2016). Democratization of
education in emerging countries. Journal of Interdisciplinary Education, 15(1), 1–36.
https://www.researchgate.net/publication/304836394_Democratization_of_Education_in_Emerging_Countries
Sari, A. R., Bonk, C. J., Zhu, M. (2019). MOOC instructor designs and challenges: What canbe learned from existing MOOCs in Indonesia and Malaysia? Asia Pacific Education
Review. 1–24. https://doi.org/10.1007/s12564-019-09618-9
Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade increase: Tracking distance
Education in the United States. Babson Survey Research Group.
http://onlinelearningsurvey.com/reports/gradeincrease.pdf
Sefa Dei G. J. & Shahjahan, R. (2008). Equity and democratic education in Ghana: Towards
a pedagogy of difference. In J. Zajda, L. Davies & S. Majhanovich (Eds.),
Comparative and global pedagogies: Equity, access, and democracy in education,(pp. 49–70). Springer.
Singh, A. B. (2018). Accessing the role of massive open online courses (MOOCs) in higher
education: Opportunities and challenges for learning. In NELTA ELT Forum 2018
volume 4 (pp. 97–113). Kathmandu: Nepal English Language Teacher’s Association.
https://www.nelta.org.np/uploads/images/files/NELTA%20ELT%20Forum%20Vol%
204%202018.pdf

44
46

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Shakya, M. & Shrestha, S. (2018). MOOCs for higher education in Nepal. Quezon City:
Foundation for Information Technology Education and Development, Inc.
Shigeta, K., Koizumi, M., Sakai, H., Tsuji, Y., Inaba, R., & Hiraoka, N. (2017). A survey ofthe awareness, offering, and adoption of OERS and MOOCs in Japan. Open Praxis,(9), 195–206. https://doi.org/10.5944/openpraxis.9.2.568
Smyrnova-Trybulska, E., Ogrodzka-Mazur, E., Szafrańska-Gajdzica, A., -Morze, N.,
Makhachashvili, R., Noskova, T., Pavlova, T., Yakovleva, O., Issa, T., & Issa, T.
(2016). MOOCs – Theoretical and practical aspects: Comparison of selected research
results: Poland, Russia, Ukraine, and Australia. Proceedings of the International
Conferences on Internet Technologies & Society (ITS), Education Technologies(ICEduTECH), and Sustainability, Technology and Education (ED571589). ERIC.
https://files.eric.ed.gov/fulltext/ED571589.pdf
Seuring, S. & Gold, S. (2011). Conducting content-analysis based literature reviews in supply
chain management. Supply Chain Management: An International Journal, 17(5),544–555. https://doi.org/10.1108/13598541211258609
SWAYAM. (2019). About SWAYAM. MRHD Government of India.
https://swayam.gov.in/about
Trehan, S., Sanzgiri, J., Li, C., Wang, R., Joshi, R. M., (2017). Critical discussions on themassive open online course (MOOC) in India and China. International Journal of
Education and Development using Information and Communication Technology,
13(2), 141–165. https://files.eric.ed.gov/fulltext/EJ1153318.pdf
UNESCO. (2015a). Incheon Declaration. Paris: UNESCO.
http://webarchive.unesco.org/20160930040522/https://en.unesco.org/worldeducation-forum-2015/incheon-declaration
UNESCO. (2015b). World Education Forum 2015 final report. Paris: UNESCO.
https://unesdoc.unesco.org/ark:/48223/pf0000243724
Upadhyay, S. & Anandpara, M. (2020). ICT best practice: MOOC courses in higher
education. 1st National conference on role of higher education in sustainable
development in India (RHESDI) 2020, 25–29.
https://wvvw.easychair.org/publications/preprint_download/Mc9X
van de Oudeweetering, K. & Agirdag, O. (2015). MOOCs as accelerators of social mobility?
A systematic review. Educational Technology & Society, 21(1), 1–11. (EJ1165986).
ERIC. https://www.j-ets.net/ETS/journals/21_1/1.pdf
Waldrop, M. M. (2013). Online learning: Campus 2.0. Nature, 495, 160–163.
https://doi.org/10.1038/495160a
World Bank. (2016, May 17). World development report 2016: Digital dividends.
http://www.worldbank.org/en/publication/wdr2016
Xuetangx (2019). About us. https://next.xuetangx.com/about
Zajda, J. (2008). Globalization, comparative education and policy research: Equity and accessissues. In J. Zajda, L. Davies & S. Majhanovich (Eds.), Comparative and global
pedagogies: Equity, access, and democracy in education, (pp. 3–14). Springer.

45
47

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Zawacki-Richter, O. & Naidu, S. (2016). Mapping research trends from 35 years of
publications in distance education. Journal of Distance Education, 37(6), 245–269.
https://doi.org/10.1080/01587919.2016.1185079
Zawacki-Richter, O., Bozkurt, A., Alturki, U., & Aldraiweesh, A. (2018). What research says
about MOOCs – An explorative content analysis. International Review of Research inOpen and Distance Learning, 19(1), 242–259.
https://doi.org/10.19173/irrodl.v19i1.3356
Zhenghao, C., Alcorn, C., Christensen, G., Eriksson, N., Koller, D., & Emanuel E. J. (2015).
Who’s benefiting from MOOCs and why. Harvard Business Review.
https://hbr.org/2015/09/whos-benefiting-from-moocs-and-why
Corresponding author: R. Pam Barger
Contact email: pam.barger@wheaton.edu

46
48

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Understanding the Characteristics of English Language Learners’
Out-of-Class Language Learning through Digital Practices
Ali Dincer
Erzincan Binali Yildirim University
Turkey47
49

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
This study aims to understand the extent to which English as a foreign language learners use
technology for their autonomous language learning beyond the classroom. With a crosssectional survey design approach, the study focuses on learner characteristics. It firstinvestigates the existing language learner profiles of 512 English major university students
concerning autonomous language learning and out-of-class technology engagement. Then,details regarding the characteristics of existing learner profiles in terms of language
proficiency, daily technology use time, a variety of digital tool use and the most beneficial
tools are outlined. Within this frame, cluster analyses suggested two clusters: moreautonomously engaged with digital tools and less autonomously engaged with digital tools.
The findings showed that more autonomously engaged students tend to have greater languagelearning proficiency than the less autonomous group. The more autonomously engaged
students also spent more time daily and used various digital tools in comparison to lessautonomous technology users. While online websites and social media were the mostfrequently used digital tools for both groups, the use of podcasts, blogs and online language
courses differed. According to the provided tool lists of learners, students benefited
significantly from social media, online websites, dictionaries, and intelligent tutoring systemapplications (apps). Online games, YouTube, Instagram and other smartphone apps, whichallow students to practice vocabulary and speak with foreigners, also had considerable
influence on language development. The study findings provide insights for language teachersaiming to extend learners’ in-class language-learning experiences beyond the confines of theclassroom.
Keywords: autonomous language learning, learning through digital practices, out-of-classlanguage learning, technology engagement

48
50

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Following the rapid development of digital technology, the worldwide digital population now
encompasses nearly 60% of the global population, with over 4.5 billion active users (Kemp,
2020). According to a recent report and statistics, the average time spent on the Internet isalmost seven hours per day, mainly on social media, watching television, listening to streaming
music and gaming. Among these activities, the use of social media is the most favored Internet
activity, and 45% of the world population is on social media, spending a minimum of two hours
each day. Almost all social media users visit social media sites via their smartphones.
Millennials are the biggest users of social media among the generations (Kemp, 2020; Moshin,2020). These statistics show that people are engaged with technology much more than before,
for information seeking, content creation, playing games, broadcasting, communication,
education, and pleasure, among other things. As a result, technology is undeniably an
indispensable part of daily life. English language, with over two billion native and non-native
speakers, serves as a communication tool by delimiting boundaries between people fromdifferent countries and creating an authentic virtual atmosphere (Ethnologue, 2019).
Unifying the potential of widespread use of digital technology and the importance of English
in the globalized world, it is necessary to understand what goes on outside the classroom tomeet the constantly changing needs of today’s digital learners. Unlike the great importance ofbridging inside and outside learning, the majority of applied linguistic research focuses on what
transpires inside the classroom as well as classroom-based application, and there remains
limited research concentrating on learning language beyond the classroom, specifically indigital settings (Chick, 2018; Lai, 2015, 2017; Reinders & Benson, 2017; Richards, 2015).
Little attention has been given to how language learners bolster their language skills and what
language learner profiles exist in learner-led informal settings (Gonulal, 2019).
This article focuses on English as a foreign language (EFL) learners’ out-of-class language
learning experiences through digital practices and reports on a survey study conducted with
university students in Turkey. Specifically, it investigates the existing characteristics of English
major language learners with a closer focus on their technology-mediated autonomous
language learning experiences. The article firstly reviews relevant research for understanding
language learners’ out-of-class digital practices and then presents the study methodology andfindings. Lastly, it discusses the findings regarding the emerging learner profiles and draws
conclusions. Therefore, it expands our understanding of what language learners do beyond theconfines of school to learn English in the digital era.
Literature Review
High proficiency of language achievement and development depends on learners’ out-of-classengagement as well as in-class engagement (Benson, 2011; Chick & Ho, 2017; Dincer &Dariyemez, 2020; Lai, 2017; Lai et al., 2015; Richards, 2015). Although great strides havebeen made in understanding foreign language learners’ out-of-class language learning practices
and gains for language development in earlier studies (e.g., Benson, 2011; Hyland, 2004; Inozu
et al., 2010; Murray & Kojima, 2007), less is known about what today’s youth do to study
language beyond the classroom in the digital era that exists today. Internet and digitaltechnology have become ubiquitous and changed the traditional teaching routines of people allover the world (Larsen-Freeman & Anderson, 2013; Mazer et al., 2007). With a transition fromtraditional definitions of computer-assisted language learning (CALL) to more modern
methods including mobile learning, the importance of digital tools and integration oftechnology into daily life has exponentially gained popularity in the language education
domain (Reinders & White, 2016).

49
51

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Learners’ online informal language experience has great value for language development in
the age of digital abundance (Chick, 2018). Recent research has emphasized that engagementin the prevalent digital practices for language learning not only plays a supporting role inlanguage development for learners, but also a complementary role for in-class language
learning (Alice & Ho, 2017; Lai, 2017; Lai et al., 2018; Nunan & Richard, 2015). Digital tools
provide meaningful and authentic language-learning opportunities for learners and might serve
as a panacea for learners who have no or limited change in exposure to authentic daily language
in an out-of-school setting (Dincer & Dariyemez, 2020; Gonulal, 2019; Lai et al., 2016;
Richards, 2015; Xodabande, 2018). Today, learner autonomy and technology-enhanced
language learning have not been isolated from each other; they are together and interconnected(Reinders & White, 2016). To excel in language, learners should have an inner capacity to beaware of their needs and desire to track their progress (Alice & Ho, 2017). Despite its
importance for language development, there is limited focus on technology-enhanced out-ofclass language learning, and there is a recurrent call for more research into autonomous
language learning through digital practices and learner characteristics (Chick, 2018; Dincer &Dariyemez, 2020; Gonulal, 2019; Lai, 2017; Lai et al., 2018; Reinders & Benson, 2017;
Reinders & White, 2016).
There is a scarcity of literature about language learners’ online informal language learning, andthe connection between autonomous language learning and digital practices in the applied
linguistics domain is not clear. Via interviews, Lai et al. (2016) elicited the perception matches
between language teachers and students regarding teacher involvement in fostering
autonomous language learning with technology beyond the classroom. They found that there
are mismatches between perceptions and teachers, although teachers have minimal
responsibility for the autonomous language development of students and play a passive role inlearners’ out-of-class use of technology for language learning as they overestimate students’skills. In later research, Chick (2018) provided research agendas on autonomous language
learning beyond the classroom and digital practices by way of a qualitative ethnographic
approach. Based on the model of Benson (2011) for searching out-of-class language learning,she mapped her language-learning experience on Duolingo (an intelligent tutoring system).
Chick found that, in her learning experience, there exist environmental factors beyond her
control and it is hard to turn a leisure activity into recreational learning. Duolingo provides a
structured pedagogy for learning and goes beyond the structural lessons. Although the locus ofcontrol is voluntary participation in the starting point, such digital practices restrict decisions
on learning at some points.
Connecting various studies (e.g., Benson, 2011; Richards, 2015), Chick concluded that
autonomous language-learning experience is a useful indicator of successful language learning,and digital practices might be tools for understanding language learning as a whole.
Emphasizing the use of any digital tool for various purposes, Lai and colleagues (2018)
recently identified three types of technological experiences of language learners engaged
outside the classroom using multiple data collection techniques. These types are instructionoriented (i.e., conscious and intentional information-seeking process for expanding knowledge
and language development), entertainment and information-oriented (i.e., not fully
unintentional and a disorganized method of obtaining and sharing information for daily life
needs and personal needs), and social-oriented (i.e., intentional use of social media to interact
and communicate with English speakers). These distinct learner profiles are important for
educators and researchers to understand language learners’ out-of-class experiences.

50
52

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

In another study, Xodabande (2018) researched 114 Iranian EFL learners’ preferences invarious digital technologies, in a context where people have some online restrictions andcensorship. Xodabande found that, despite the restrictions, EFL learners are highly engaged
with technology for foreign language, whereby electronic dictionaries, Internet sites and films
are the most favored digital tools. Gender was also found to be a significant factor. While malestudents engage in online games to facilitate learning, female students are more inclined tolisten to English music. In more recent research, Kuznetsova and Soomro (2019) surveyed 137
foreign language learners’ out-of-class Web 2.0 practices for learning various languages,
including English. They found that video sharing websites and social networking sites are themost widely used digital technologies and male students’ digital practices are significantly more
frequent than females, verifying some earlier research (Cai et al., 2017; Xodabande, 2018).
Despite significant progress in understanding the nature of language learners beyond theclassroom, the literature has been invaded by the umbrella term “out-of-class language
learning”, which comprises all activities and includes the technology. The extant research does
not provide a clear picture of language learners’ digital practices without teacher guidance anddirection. There is a need for research on autonomous language learning with digital practices
beyond the classroom. One certain aspect of out-of-class language learning with technology isthat “there is a great range of diversity in environments, intentionality, interest, structure, andduration” (Chick, 2018, p.76).
Improving understanding of characteristics of language learners regarding autonomy andtechnology engagement and investigating their engagement in the digital practices for language
learning might serve as a master key to unlock innovative techniques for classroom pedagogy
and bind formal to informal learning (Chick, 2018; Kuzetsova & Soomro, 2019; Lai & Gu,
2011; Lai et al., 2015). Considering the affordances of language engagement through digitalpractices beyond the classroom and the scarce research on autonomous language learning
through digital practices, there is much yet to be discovered on specific language learners’
distinct characteristics in this unexplored field of research. By way of a survey approach, thisstudy specifically focuses on learning English beyond the classroom with digital tools andstudents’ own initiatives, and aims to understand to what extent EFL learners use technologicaltools for their self-directed language learning beyond the classroom. With a particular focus on
language learners’ characteristics, the following research questions guided the study:
1. What different language learner profiles exist among English language learners in termsof autonomous language learning and out-of-class technology engagement?
2. What are the characteristics of language learner profiles in terms of language
proficiency, daily technology use, and digital tools used?
Theoretical Framework
This survey research was grounded on the main tenets of modern motivation theory, selfdetermination theory [SDT] (Deci & Ryan, 1985; Ryan & Deci, 2017). SDT provides anunderstanding of the psychological and social foundations of autonomous learning in life (Louet al., 2018). Briefly, it suggests that people have innate psychological needs (i.e., autonomy,competence and relatedness), and these universal needs are met through people’s interactions
in a social context (Deci & Ryan, 1985; Ryan & Deci, 2017). Within language education, thesatisfaction of the needs is crucial for motivated engagement, and results in more autonomous
language learning and engagement (Noels et al., 2019).

51
53

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Despite the increased research on what happens in the classroom, limited research focus has
been placed on understanding what happens beyond the classroom, specifically language
learning through digital practices within the SDT framework. Connecting the SDT and therecent perspective on the technological experiences of language learners (Lai et al., 2018), thisstudy hypothesizes that autonomous language learning is positively linked to technology
engagement in language learning. Accordingly, the students who feel more autonomy inlanguage learning are prone to be more engaged with language learning beyond the classroom
through digital tools. Then, investigating the learner profiles will shed light on future research
agendas on autonomous language learning through digital practices in the language-learning
domain.
MethodResearch Design
A cross-sectional survey design was adopted to investigate the nature of language learners’
out-of-class language learning experiences. According to Creswell (2012), this is the mostpopular form of survey design in educational research. The researcher collects data to makeinferences about a particular population of interest at a given point and thereby takes snapshotsof the population (Creswell, 2012; Hall, 2008). With a non-experimental approach, this study
focused on emerging profiles of English language learners’ out-of-class language learning
through digital practices. It investigated their profiles from various perspectives such aslearners’ perceived language proficiency, daily time spent for language learning, and variety
of digital tools in language learning.
Study Context
The study was conducted with English major students studying in the English Language
Teaching (ELT) and English Language Literature (ELL) departments of three state universities
in Turkey. The students who graduate from these departments with a BA diploma and certain
training certificates might work as English teachers in state schools after taking teacher
placement tests. It was expected that these students’ experiences with digital tools would be
different from the students in the non-English major departments where mother tongue is themedium language.
Participants
512 university-level language learners (n = 362; 70.7% female) whose ages ranged from 17 to
32 (M = 20.59; SD = 2.46) participated in the study. They were enrolled in the two English
majors related to learning and teaching English at a tertiary level in Turkey (ELT = 304; ELL= 208). Students were from various classes concerning their majors, and a majority of themwere in the first year of their major (n = 200; 39.1%). Their overall perceived language
proficiency from receptive (reading and listening) and productive skills (writing and speaking)changed from A2 to C2 in accordance with six levels (A1, A2, B1, B2, C1, C2) of the Common
European Framework Proficiency Matrix (n = 185; mostly B2, 36.1%).
Instruments
A survey form including demographic details (i.e., gender, major, and year in the university,
perceived level of proficiency), a language learning through digital practices questionnaire,
psychometric scales of learner autonomy and out-of-class technology engagement were usedas the data collection instruments.

52
54

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Language learning through digital practices. The questionnaire consisted of three questions
regarding students’ digital practices. The first question was about daily time spent using
technology, from the least (one hour) to high (four hours). The second question was about
digital tools students might use while learning on their own. The students chose what digital
tool/s they used to learn a foreign language on their own from the given 10 most cited digital
tools in the relevant literature (e.g., Benson, 2011; Lai, 2017; Richards, 2015). The third
question was in the open-ended format, about the most beneficial digital tool/s used withregularity and their reasons of preferences.
Learner autonomy. The scale was adapted from Nakata (2011) to measure the students’autonomous language-learning experiences. It was a five-point Likert scale from 1 (Strongly
Disagree) to 5 (Strongly Agree). It had eight items concerning out-of-class learning (e.g., I
decide what to learn outside the classroom; α = .77). Higher mean scores from the items
indicate higher agreement and autonomy levels.
Out-of-class technology engagement. The scale (Lai et al., 2018) assessed the degree to whichlearners engage in language learning by using digital tools with a five-point Likert scale from
1 (Strongly Disagree) to 5 (Strongly Agree). The scale started with the prompt “I use digital
tools outside the classroom, mainly …” and was followed by 11 items. The scale had three
dimensions: instruction-oriented technological experience had three items (e.g., to help mememorize the vocabulary and grammar; α = .70); the entertainment- and information-oriented
experience had five items (e.g., to use the language to go after personal interest; α = .74); andthe social-oriented technological experience dimension had three items (e.g., to connect withnative speakers or other learners of the foreign language; α = .80). In each dimension, a higher
mean indicated stronger agreement and engagement.
Data Collection and Analysis
The data were collected using a paper-based survey in spring 2019. After departmental
approval was obtained for data collection, the students were invited to participate in the study.
The students were informed about the purpose of the study and instructed about the meaningof learning English with digital tools with examples by the collaborators. They were also
informed about anonymity and encouraged to answer the open-ended question. It tookapproximately 15 minutes to complete the survey.
To answer the research questions, first, the surveys were screened, and those with a significant
amount of missing data (e.g., not completing one of the scales) and outliers (i.e., strongly
agreeing with all items or strongly disagreeing with all items) were excluded (25 out of 537).
After the reliability analysis, a cluster analysis followed, allowing researchers to extract
learners’ out-of-class technology profiles. The analysis enabled the researcher “to create a new
categorical variable that minimizes the amount of variation within categories” (Staples &Biber, 2015, p. 243). Following the steps of the Staples and Biber (2015) analysis, ahierarchical cluster analysis was run, with the Z-scores of two main variables (i.e., out-of-classtechnology experience and autonomous language learning). The analysis suggested a twocluster solution. Based on the determined language learner profiles, both quantitative andqualitative data from the survey were descriptively analyzed. For the quantitative data analysis,
the SPSS 22.0 packet program was used. In the qualitative data analysis, NVivo software
version 12 was used to review word frequencies and to produce a words cloud. In thepresentation of the quantitative findings, descriptive tables were used. For the qualitative data,
students’ excerpts from the open-ended questions were provided with the capital letter of thecluster and the student data identification number (i.e., M123).

53
55

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Findings
Learner Profiles in Out-of-Class Language Learning Through Digital Practices
The first research question concerned existing language learner profiles in terms of autonomous
language learning and out-of-class technology engagement. The hierarchical cluster analysis
suggested a two-cluster solution (Cluster 1, n = 330; Cluster 2, n = 182). The findings arepresented in Table 1.
Table 1: Comparison of learner profiles in terms of autonomous language learning and outof-class technology engagement
Main variables

Sub-variables

Autonomous
language learning

Learnerautonomy

Out-of-classtechnology
engagement

Instructionoriented
Entertainment
and informationoriented
Social-oriented

Cluster 1
M
4.12

SD
.44

Cluster 2
M
SD
3.40
.57

4.11

.50

3.38

.83

4.25

.44

3.44

.63

4.11

.71

2.99

.84

The results indicated that the students in Cluster 1 highly agreed with the scale items and were
thereby named ‘More autonomously engaged with digital tools’ (M ≥ 4.00 = Agree). The
students in Cluster 2 had a moderate level of agreement and were named ‘Less autonomously
engaged with digital tools’ (M < 4.00 = Moderately Agree).
After the determination of the clusters, the demographic characteristics of the groups wereanalyzed. According to the descriptive statistics, both clusters had similar demographic
characteristics, such as age (Cluster 1: Mage = 20.67, SD = 2.42; Cluster 2: Mage = 20.43; SD =2.53) and department distribution.
Two groups of descriptive findings are presented in Table 2. According to Table 2, most of theparticipants in both groups were females (Min. = 64.3%), from the ELT department (Min. =57.1%), and first year of study (Min. = 39.4%).
Table 2: Demographics of the clusters
Demographics
Gender
Major
Year inuniversity

Male
FemaleELT
ELL
Preparatory
First year
Second year
Third year
Fourth year

More autonomously
engaged (n = 330)
f
%
85
25.8
245
74.2
200
60.6
130
39.4
55
16.7
122
37.0
43
13.0
39
11.8
71
21.5

54
56

Less autonomously
engaged (n = 182)
f
%
65
35.7
117
64.3
104
57.1
78
42.9
40
22.0
78
42.9
16
8.8
21
11.5
27
14.8

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

In sum, there were two types of learner profiles in terms of digital practices and autonomous
language learning beyond the classroom. The clusters had similar demographic details
regarding frequencies in the variables (i.e., gender, major and year at the university).
Characteristics of language learner profiles
The second research question focused on understanding the nature of existing language learnerprofiles in terms of language proficiency, daily time spent, variety of digital tools used andmost beneficial tools.
Language proficiency. First, the clusters’ perceived language proficiency levels werecompared in terms of receptive and productive language skills. Table 3 indicates thefrequencies and percentages in terms of perceived language proficiency levels in receptive
skills: reading and listening.
Table 3: Comparison of clusters in terms of receptive skills
Receptive skills
Reading

Listening

A1 (Beginner)
A2 (Elementary)
B1 (Intermediate)
B2 (Upper Intermediate)C1 (Advanced)
C2 (Proficiency)
A1 (Beginner)
A2 (Elementary)
B1 (Intermediate)
B2 (Upper Intermediate)C1 (Advanced)
C2 (Proficiency)

More autonomously
engaged (n = 319)
f
%
0
0.0
8
2.5
32
10.0
111
34.8
108
33.9
60
18.8
3
0.9
33
10.3
72
22.6
91
28.5
76
23.8
44
13.8

Less autonomously
engaged (n = 181)
f
%
0
0.0
10
5.5
44
24.3
78
43.1
39
21.5
10
5.5
13
7.2
33
18.2
42
23.2
48
26.5
37
20.4
8
4.4

According to Table 3, students who are more autonomously engaged with digital tools havehigher perceived receptive language proficiency levels than those less autonomously engaged
with digital tools (i.e., reading and speaking = B2 and C1). Over half of the students in bothgroups (i.e., Min. = 51%) perceived their levels to be over level B1.
Students’ comparisons in terms of perceived levels in productive skills are presented in Table
4, which indicates the frequencies and percentages of the productive skills: writing andspeaking.

55
57

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 4: Comparison of clusters in terms of productive skills
Productive skills
Writing

Speaking

A1 (Beginner)
A2 (Elementary)
B1 (Intermediate)
B2 (Upper Intermediate)C1 (Advanced)
C2 (Proficiency)
A1 (Beginner)
A2 (Elementary)
B1 (Intermediate)
B2 (Upper Intermediate)C1 (Advanced)
C2 (Proficiency)

More autonomously
engaged (n = 319)
f
%
0
0.0
28
8.8
73
23.0
98
30.8
88
27.7
31
9.7
12
3.8
34
10.7
102
32.0
74
23.2
71
22.3
26
8.2

Less autonomously
engaged (n = 181)
f
%
0
0.0
28
15.5
56
30.9
54
29.8
38
21.0
5
2.8
14
7.7
52
28.7
46
25.4
38
21.0
25
13.8
6
3.3

In parallel to the findings of the receptive skills, the students in the more autonomously engaged
with digital tools cluster showed higher perceived proficiency levels in productive skills than
the students in the second group (i.e., more autonomously engaged = B2 and C1). Thesestudents had lower proficiency in writing skills in comparison to speaking skills (moreautonomously engaged = Max.writing 30.8% in B2 level; more autonomously engaged =Max.speaking = 32.0% in B1 level). While the highest majority of the more autonomously
engaged students perceived their speaking skills at B1 (i.e., more autonomously engaged =speaking 32.0%), the students in the less autonomously engaged with digital tools group
perceived themselves at level A2 (i.e., less autonomously engaged = speaking 28.7%).
Daily time spent. Second, the clusters were compared in terms of their daily time spent using
digital tools in language learning, and findings are presented in Table 5.
Table 5: Comparison of clusters in terms of daily time spent
Daily timespentMin. 1 hour
Min. 2 hours
Min. 3 hours
Min. 4 hours

More autonomously engaged(n = 330)
f
%
94
28.5
140
42.4
62
18.4
34
10.3

Less autonomously engaged(n = 181)
f
%
82
45.3
70
38.7
25
13.8
4
2.2

The table shows that the students differed in terms of daily time spent on digital tools.
According to the table, the majority of students in the more autonomously engaged with digital
tools group spent a minimum of one to two hours (i.e., 70.9%). The great majority of students
in the less autonomously engaged group also spent a minimum of one to two hours learning
English digitally on a daily basis (i.e., 84.0%). Further, the table shows that students in the
more autonomously engaged group spent much more time on digital language learning with aminimum of two hours than the less autonomously engaged group.

56
58

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Digital tool use. Fourth, the clusters were compared in terms of the variety of digital tools thatlearners use, as shown in Table 6.
Table 6: Comparison of clusters in terms of tool variety
Tool variety

More autonomously engaged(n = 330)
f
%
3
0.9
26
7.9
59
17.9
64
19.4
63
19.1
48
14.5
40
12.1
16
4.8
8
2.4
3
0.9

1 tool
2 tools
3 tools
4 tools
5 tools
6 tools
7 tools
8 tools
9 tools
10 tools

Less autonomously engaged(n = 182)
f
%
6
3.3
47
25.8
37
20.3
41
22.5
27
14.8
14
7.7
8
4.4
2
1.1
0
0.0
0
0.0

Table 6 shows that the students in the more autonomously engaged group differ more in termsof digital tool variety than those less autonomously engaged. While the majority of students inthe more autonomously engaged group use a minimum of five to 10 different digital tools (i.e.,
73.2%), the majority of students in the less autonomously engaged group use one to four digital
tools for language learning (i.e., 71.9%).
In addition to the digital tool variety comparison, students’ preferences for most frequently
used digital tools were determined. The comparison of clusters with regards to the tool
frequency is presented in Table 7.
Table 7: Comparison of clusters in terms of tool use frequency
Tools
1. Social media
2. Dictionaries
3. Websites
4. Grammar &spelling checkers
5. Intelligent
tutoring systems
6. Blogs
7. Forums
8. Podcasts
9. Online courses
10. Automatic
speech recognition

Moreautonomously
engaged (n = 330)
f
%
297
90.0
285
86.4
223
67.6
169
51.2
159

48.2

118
100
93
88

35.8
30.3
28.2
26.7

69

20.9

Tools
1. Social media
2. Dictionaries
3. Websites
4. Grammar &spelling checkers
5. Intelligent
tutoring systems
6. Online courses
7. Forums
8. Blogs
9. Automatic speech
recognition
10. Podcasts

57
59

Less
autonomously
engaged (n = 182)
f
%
156
85.7
151
83.0
91
50.0
64
35.2
52

28.6

39
35
29
28

21.4
19.2
15.9
15.4

25

13.7

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

While the first five tools listed in the table were the most commonly used digital tools by bothgroups, the remaining five tools differed between the groups. While social media, online
dictionaries and online websites for language learning were the most frequently used digital
tools for both groups, the use of podcasts and online language courses differed.
Most benefited digital tools. Fifth, students’ answers to the open-ended question about themost beneficial digital tools for learning English were descriptively analyzed (N = 307). Whileanswering the question, some of the students listed more than one digital tool as mostbeneficial, thereby making frequencies by tool higher than the number of participants. It should
also be noted that two extra tools emerged from the data: online games and smartphone apps,
allowing for speaking with native speakers for a fee. The findings are shown in Table 8.
Table 8: Comparison of clusters in terms of most beneficial digital tools
Tools
1.Social media
2.Websites
3.Dictionaries
4. Intelligent
tutoring systems
5. Online courses
6. Online games
7. Grammar &spelling checkers
8. Blogs
9. ForumsMoreautonomously
engaged (n = 215)
f
%
89
41.4
50
23.3
49
21.4
42
19.5
18
13
9

8.4
6.1
4.2

7
6

3.3
2.8

Tools
1. Social media
2. Websites
3. Dictionaries
4. Intelligent
tutoring systems
5. Online courses
6. Podcasts
7. Forums

8. Blogs
9. Grammar &spelling checkers
10. Podcasts
5
2.3
10. Online games
11. Apps-Speaking
5
2.3
11. Apps-Speaking
12. Automatic2
0.9
12. Automatic
speech recognition
speech recognition
Note. Values might not end in 100% due to rounding.

Lessautonomously
engaged (n = 92)
f
%
32
34.8
23
25.0
20
21.7
14
15.2
5
5
4

5.4
5.4
4.4

2
2

2.2
2.2

2
1
0

2.2
1.1
0.0

According to the most beneficial tool lists, students’ preferences provided similar findings to
the previous digital tool frequency list table. Social media, online websites, dictionaries andintelligent tutoring system apps were the most beneficial tools for all students. Different fromthe tool lists, some students indicated the benefits of online games and smartphone apps, whichallow them to speak with native English speakers and people who use English forcommunication.
In addition to frequencies, to assess specific tools among all students’ answers, a word cloud
was generated, with a restriction of words with a minimum length of five letters and the 30
most frequently used words with stemmed words. The results are shown in Figure 1.

58
60

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Figure 1: Word cloud
The word cloud formed from 307 students’ open-ended responses indicated that, in addition tosome broad tools stated, students also use specific tools. These tools are YouTube (online
video-sharing platform and social media tool), BBC English (online language service of BBC
World Service), Tureng (a bilingual online Turkish-English dictionary), Instagram (a social
networking service), Duolingo and Memrise (intelligent tutoring systems for language
learning).
The students emphasized the many benefits of learning English with digital tools. Digital tools
allow learners to access the content anytime they want using their cellphones. On this issue,
one student (M159) said: “I think YouTube is the most useful for me because I accesseverything about learning the language with easily [sic].” Another participant (L91) added: “Ihave benefited much with social media websites because I always can access it on my phone
all the time.”
Digital tools allow learners to practice numerous skills at once. L4 said: “YouTube, because I
can improve more skills, such as listening, reading, speaking, compared to other tools.” Anotherstudent said: “Podcasts, because they enhanced both my speaking and listening skills.” (L47)
In addition to language development gains, students emphasized that language learning withdigital tools is a win-win process. While being entertained, learning comes as a bonus, andlearners broaden their horizons. On this issue, M81 said: “YouTube and Instagram. Especiallymemes, I follow on Instagram because it entertains me while learning.” Focusing on the roleof BBC English regarding language development, L90 pointed out that: “In this way, I not only
improve my listening skills but also I know what happened all around the world.”
Digital tools allow learners to access high-quality content in accordance with their own tastes
and repeat the content anytime they want. Regarding this topic, L66 stated that: “BBC English

59
61

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

learning is the best. It is very qualified, and it has many good contents.” M50 said: “Duolingo
is very beneficial because you can repeat until you learn.”Moreover, digital tools allow students to practice the language in an interactive, authentic
language learning setting and learn the daily language and colloquial expressions that are
difficult to gather in-class. Emphasizing the social interaction version of YouTube, student
M196 indicated that “Because on this platform, I watch dozens of videos, and we can discussin the comments with other people.” Focusing on the social role of online games, H45 said:“Games, because I hear original sentences which I could not hear before and interact withnatives [English] in Clanset.”
Some of the students also highlighted the complementary role of digital tools for in-classeducation. For instance, M75 indicated: “I benefit much from online websites. My lessons [inclass] are not enough to study learning English.” Emphasizing the limited in-class course hours,
L83 stated: “I have benefited from Blogs much because to improve my English, I have to use
listening texts.”
Discussion
This study has explored the learner profiles in self-directed language-learning initiatives
through digital practices and mapped the nature of university students’ out-of-class English
language engagement using a survey design. Grounded on the main tenets of SDT, it was
hypothesized that autonomous language learners are active learners outside the classroom, as
well as in the classroom. They successfully self-direct their language learning without teacher
direction. Then, the study connected learner autonomy and out-of-class technology
engagement for learning English and aimed to investigate distinct characteristics of language
learners. Two research questions guided the study, and the findings for each research question
have been outlined.
To answer the research questions, first, a hierarchical cluster analysis was conducted,
suggesting two distinct profiles in terms of autonomous language learning through digitalpractices: “More autonomously engaged” versus “Less autonomously engaged” students. Thisfinding verified the hypothesis of the study. As expected, the students who feel greater feelingsof autonomy in their learning are also the ones who are highly engaged in the learning process.
In concurrence with the relevant literature, the findings showed that these two variables areclosely associated and students with high levels of learner autonomy tend to have strong outof-class technology engagement (Chick, 2018; Lai & Gu, 2011; Lai et al. 2015, 2016; Reinders& White, 2016). Further, it should be noted that, unlike the earlier literature suggesting that
EFL learners have some problems taking responsibility in their learning process and needsignificant teacher guidance (Eksi & Aydin, 2013; Inozu et al., 2010), the means for bothgroups’ out-of-class digital language-learning experience and autonomous language learning
were relatively high. This outcome corroborates the recent studies about the findings of thetechnology-enhanced language practice, which suggest that students in the digital age takemore initiative in their language learning despite waiting from their teachers (Haidari et al.,2019; Lai et al., 2018). This might be related to the rapid development of digital tools,
smartphones and apps in the last decade (Jurkovič, 2019).
Based on the emerged learner profiles, a number of comparisons were conducted between thegroups to understand the different learner profiles’ engagement with a digital tool. Regarding
the contrast within the perceived language proficiency levels, students who are more

60
62

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

autonomously engaged with digital tools tend to have a higher proficiency both in receptive
and productive language skills than the less autonomously engaged students. In parallel toliterature (e.g., Dincer & Yesilyurt, 2017), students have lower proficiency levels in productive
skills in EFL contexts. According to daily time spent and variety of used tools, moreautonomously engaged students spend greater time daily for learning English online and use awider variety of digital tools than the less autonomously engaged students. As Lai and Gu(2011) stated, perceived language proficiency is associated with learners’ use of technology toseek language-learning resources and opportunities beyond the classroom. Students who havestudied the language for a long time and have higher proficiency levels tend to spend moretime daily and assume more personal responsibility (Orhon, 2018).
With regard to the most frequently used digital tools, social media, online dictionaries, websites
for language learning, grammar and spelling checkers, and intelligent tutoring systems were
found to be the most popular for both groups, which is consistent with a number of studies (Caiet al., 2017; Kuznetsova & Soomro, 2019). Despite the affordances for language development,
tools such as blogs, forums and automatic speech recognition are the least popular. Althoughblogs and forums serve as social-oriented technological experience for learners, they are theleast popular activities in earlier studies (Lai & Gu, 2011; Lai et al., 2018). In addition, there
is limited research on the use of automatic speech recognition tools in language learning, as
they have only recently gained popularity due to the prevalent IOS and Android applications.
With the improvements in accurate speech recognition, automatic speech recognition
technology has become ubiquitous and important for fostering learner autonomy (McCrocklin,
2016). In conjunction with the frequency of use of the tools, students in both groups benefited
most from social media, websites for language learning and online dictionaries. Learners
named language-learning websites like BBC English learning, social media sites such as
YouTube and Instagram, and digital games as very helpful tools. It was shown that English
major students are more aware of the benefits of the tools and use these effectively for language
development. In contrast, Xodabande (2018) found such online websites and computer games
to be perceived as less effective tools in enhancing language development. However,
Xodabande’s results were country-specific, and most social media sites are filtered; these tools
do not require much effort to push learners to use them actively. The majority of tools provide
receptive activities and entertainment, and information-oriented activities are more common indigital language-learning practices (Jurkovič, 2019; Lai, 2015, 2017; Lai et al., 2015, 2018).
Furthermore, the findings of this study suggest that online digital tools allow learners topractice skills with ease at all times. In parallel to the entertainment- and information-oriented
technological experience (Lai et al., 2018), students share or access information useful in daily
life while being entertained. In addition, this unintentional process might turn into theincidental acquisition of the language, and naturalistic learning might occur (Sockett, 2014).
Online digital tools allow students to practice English in an interactive setting focused on daily
language. Most digital tools, especially social media, provide authentic language and increase
learners’ exposure to target language outside the classroom, especially in EFL settings (Hyland,
2014; Richards, 2015; Xodabande, 2018). Such tools might be used to compensate for the
deficiencies of formal education, such as limited course hours and oral practice (Lai et al.,2016; Richards, 2015; Xodabande, 2018). As Lai et al. (2016) emphasized, learning language
through digital practice beyond the classroom might serve a compensatory role for limited
course time or for practicing lesser known or unclear aspects that are not addressed in theformal education.

61
63

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

The limitations of this study involve the study participants and data collection methodology.
First, although two learner profiles emerged, both groups had relatively high means in language
learning with technology outside the classroom. This might be related to the study group, as
the participants were students in the English major. English major students might have greater
needs as well as more self-determined motivation levels than learners who study the language
for more external reasons, such as passing an exam or a career promotion (Dincer & Yesilyurt,2017). In this vein, the other groups of students in different branches might be a strategic new
focus of research for further inquiry. Second, this study adopted a survey approach togeneralize and understand what transpires outside the classroom in terms of language learningwith digital tools. Although this methodology fit the research goals well, it lacked presentationof the causal relationships between the reasons for use and specific tools, and says little about
predominant language use in all digital activities. Then, one-on-one interviews are needed forin-depth analysis and understanding of how these digital tools helped learners to learn English
beyond the classroom. It might also be interesting to adopt a mixed-method or longitudinal
design to understand the relationship and activities, including language use beyond theclassroom (Jurkovič, 2019). As Reinders and Benson (2017) suggested, asking learners to keep
diaries or reflective journals to track their engagement and individual strategy use in-depth ona daily basis might illuminate the specific tools used in this lesser known terrain of applied
linguistics.
Conclusion
The study findings provide further understanding of the nature of language learners’ out-ofclass language learning experiences with digital tools. The research provided findings in which
tools are used by more autonomously engaged and less autonomously engaged learners. The
findings reveal that learners are highly engaged with digital tools, and that these digital tools
afford for learners to individualize and monitor their language development. Additionally, thestudy highlights the necessity for language teachers to homogenize students’ out-of-classexperiences and in-class language learning. With the divergence of digital tools and the use ofsmartphones in daily life, the boundaries between in-class and out-of-class have gradually
blurred (Ma, 2017). Therefore, it seems imperative for teachers to guide their students’ learning
outside the classroom and link in-class teaching to their students’ out-of-class activities by
understanding their students’ engagement with digital tools (Reinders & Benson, 2017).
Research on what transpires beyond the classroom with digital tools for language learning
remains a raw area, and there are many questions waiting to be answered. Any research attempt
to understand language learners’ practices outside the classroom in the digital era would assist
in developing thorough understanding of the language-learning process.
Acknowledgements
The author thanks Hakan Demiroz (PhD) and Tuba Isik (PhD Student) for their data collection
assistance.

62
64

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Benson, P. (2011). Language learning and teaching beyond the classroom: An introduction.
In P. Benson & H. Reinders (Eds.), Language learning and teaching beyond theclassroom: Theoretical and practical perspectives (pp. 7–16). Palgrave Macmillan.
Cai, Z., Fan, X., & Du, J. (2017). Gender and attitudes toward technology use: A metaanalysis. Computers & Education, 105, 1–13.
https://doi.org/10.1016/j.compedu.2016.11.003
Chan, H. W. (2016). Popular culture, English out-of-class activities, and learner autonomy
among highly proficient secondary students in Hong Kong. Universal Journal of
Educational Research 4(8), 1918–1923. https://doi.org/10.13189/ujer.2016.040823
Chik, A. (2018). Learner autonomy and digital practices. In A. Chik, N. Aoki, & R. Smith(Eds.), Autonomy in language learning and teaching: New research agenda (pp. 73–
92). Palgrave Pivot. https://doi.org/10.1057/978-1-137-52998-5-5Chik, A., & Ho, J. (2017). Learn a language free: Recreational learning among adults.
System, 69, 162–171. https://doi.org/10.1016/j.system.2017.07.017
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating
quantitative and qualitative research (4th ed.). Pearson Education.
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in humanbehavior. Plenum Press.
Dincer, A., & Dariyemez, T. (2020). Proficient speakers of English as a foreign language: A
focus-group study. IAFOR Journal of Education: Language Learning in Education,
8(1), 83–99. https://doi.org/10.22492/ije.8.1.05
Dincer, A., & Yesilyurt, S. (2017). Motivation to speak English: A self-determination theory
perspective. PASAA: Journal of Language Teaching and Learning in Thailand, 53(1),1–25. https://doi.org/10.14456/pasaa.2017.1
Eksi, G., & Aydin, H. (2013). What are the students doing out there? An investigation of outof-classs language learning activities. Abant Izzet Baysal University Journal of Social
Sciences, 13(2), 191–210.
Ethnologue, (2019). What are the top 200 most spoken languages? Retrieved fromhttps://www.ethnologue.com/statistics/size
Gonulal, T. (2019). The use of Instagram as a mobile-assisted language learning tool.
Contemporary Educational Technology, 10(3), 309–323.
https://doi.org/10.30935/cet.590108
Haidari, S. M., Yanpar Yelken, T., & Akay, C. (2019). Technology-enhanced self-directedlanguage learning behaviors of EFL student teachers. Contemporary Educational
Technology, 10(3), 229–245. https://doi.org/10.30935/cet.590003
Hall, J. (2008). Cross-sectional survey design. In P. J. Lavrakas (Ed.), Encyclopedia of survey
research methods (pp.172–173). Sage. https://doi.org/10.4135/9781412963947
Hyland, F. (2004). Learning autonomously: Contextualizing out-of-class English language
learning. Language Awareness, 13(3), 180–202.
https://doi.org/10.1080/09658410408667094

63
65

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Inozu, J., Sahinkarakas, S., & Yumru, H. (2010). The nature of language learning experiences
beyond the classroom and its learning outcomes. US-China Foreign Language, 8(1),14–19.
Jurkovič, V. (2019). Online informal learning of English through smartphones in Slovenia.
System, 80, 27–37. https://doi.org/10.1016/j.system.2018.10.007
Kemp, S. (2020). Digital in 2020. Retrieved from https://wearesocial.com/digital-2020
Kuznetsova, N., & Soomro, K. A. (2019). Students’ out-of-class Web 2.0 practices in foreign
language learning. Journal of Education and Educational Development, 6(1), 78–94.
https://doi.org/10.22555/joeed.v6i1.2713
Lai, C. (2015). Modeling teachers’ influence on learners’ self-directed use of technology for
language learning outside of the classroom. Computer & Education, 82(1), 74–83.
https://doi.org/10.1016/j.compedu.2014.11.005
Lai, C. (2017). Autonomous language learning with technology: Beyond the classroom.
Bloomsbury.
Lai, C., & Gu, M. (2011). Self-regulated out-of-class language learning with technology.
Computer Assisted Language Learning, 24(4), 317–335.
https://doi.org/10.1080/ 09588221.2011.568417
Lai, C., Hu, X., & Lyu, B. (2018). Understanding the nature of learners’ out-of-classlanguage learning experience with technology. Computer Assisted Language
Learning, 31(1–2), 114–143. https://doi.org/10.1080/09588221.2017.1391293
Lai, C., Yeung, Y., & Hu, J. (2016). University student and teacher perceptions of teacher
roles in promoting autonomous language learning with technology outside theclassroom. Computer Assisted Language Learning, 29(4), 703–723.
https://doi.org/10.1080/09588221.2015.1016441
Lai, C., Zhu, W., & Gong, G. (2015). Understanding the quality of out-of-class English
learning. TESOL Quarterly, 49(2), 278–308. https://doi.org/10.1002/tesq.171
Larsen-Freeman, D., & Anderson, M. (2013). Techniques and principles in language
teaching (3rd ed.). Oxford University.
Lou, N. M., Chaffee, K. E., Vargas Lascano, D. I., Dincer, A. & Noels, K. A. (2018).
Complementary perspectives on autonomy in self-determination theory (SDT) andlanguage learner autonomy (LLA). TESOL Quarterly, 52(1), 210–220.
https://doi.org/10.1002/tesq.403
Ma, Q. (2017). A multi-case study of university students’ language-learning experience
mediated by mobile technologies: A socio-cultural perspective. Computer Assisted
Language Learning, 30(3–4), 183–203.
https://doi.org/10.1080/09588221.2017.1301957
Mazer, J. P., Murphy, R. E., & Simonds, C. J. (2007). I’ll see you on "Facebook": The effectsof computer-mediated teacher self-disclosure on student motivation, affective
learning, and classroom climate. Communication Education, 56(1), 1–17.
https://doi.org/ 10.1080/03634520601009710
McCrocklin, S. M. (2016). Pronunciation learner autonomy: The potential of automatic
speech recognition. System, 57, 25–42. https://doi.org/10.1016/j.system.2015.12.013
Moshin, M. (2020). 10 social media statistics you need to know in 2020. Retrieved fromhttps://www.oberlo.com/blog/social-media-marketing-statistics

64
66

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Murray, G., & Kojima, M. (2007). Out-of-class language learning: One learner’s story. In P.
Benson (Ed.), Learner autonomy 8: Insider perspectives on autonomy in language
learning and teaching (pp. 25–40). Authentik.
Nakata, Y. (2011). Teachers' readiness for promoting learner autonomy: A study of Japanese
EFL high school teachers. Teaching and Teacher Education, 27(5), 900–910.
https://doi.org/10.1016/j.tate.2011.03.001
Noels, K. A., Lou, N. M., Vargas Lascano, D. I., Chaffee, K. E., Dincer, A., Zhang, Y. S. D.,& Zhang, X. (2019). Self-determination and motivated engagement in language
learning. In M. Lamb, K. Csizér, A. Henry, & S. Ryan (Eds.), The Palgrave handbookof motivation for language learning (pp. 95–115). Palgrave Macmillan.
Nunan, D., & J. C. Richards (Eds.) (2015). Language learning beyond the classroom.
Routledge.
Orhon, Y. (2018). An investigation of out-of-class language activities of tertiary-level EFL
learners. Education Reform Journal, 3(1), 1–14.
https://doi.org/10.22596/erj2018.0301.1.14
Reinders, H., & Benson, P. (2017). Research agenda: Language learning beyond the classroom. Language Teaching, 50(04), 561–578.
https://doi.org/10.1017/S0261444817000192
Reinders, H., & White, C. (2016). 20 years of autonomy and technology: How far have wecome and where to next? Language Learning & Technology, 20(2), 143–154. https://doi.org/10125/44466
Richards, J. C. (2015). The changing face of language learning: Learning beyond theclassroom. RELC Journal, 46(1), 5–22. https://doi.org/10.1177/0033688214561621
Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs inmotivation, development, and wellness. The Guilford Press.
Sockett, G. (2014). The online informal learning of English. Palgrave Macmillan.
Staples, S., & Biber, D. (2015). Cluster analysis. In L. Plonsky (Ed.), Advancing quantitative
methods in second language research (pp. 243–274). Routledge.
Xodabande, I. (2018). Iranian EFL learners' preferences of different digital technologies forlanguage learning beyond the classroom. International Journal of Education &Literacy Studies, 6(3), 20–31. https://doi.org/10.7575/aiac.ijels.v.6n.3p.20
Corresponding author: Ali Dincer
Contact email: adincer@erzincan.edu.tr

65
67

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

66
68

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Beyond the Basics: Adapting an Open Textbook to Accommodate a Flipped Class
Ariana Eichelberger
University of Hawaii at Manoa,
USA
Hong Ngo
University of Hawaii at Manoa
USA

67
69

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
Through a collaborative effort between an instructional design team and the Biology
Department at the University of Hawaiʻi at Mānoa, the OpenStax Biology 2e open, online
textbook was modified with new features to improve student engagement and learning
outcomes. This study investigated students’ perceptions of the customized textbook. A surveyof 22 questions was completed by 446 students using the textbook as a part of an introductory
Biology course. Changes made to the textbook were well received by the majority of students
and suggestions were made for improvements. Current and future revisions to the textbook are
discussed.
Keywords: OER, OpenStax, biology, flipped classroom

68
70

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Assigning university students readings to complete outside of class is not new. Outside
readings are an important way for students to learn, as undergraduate courses typically have
limited in-class time (Bergmann & Sams, 2012; Ryan, 2006). Students who complete readings
prior to class tend to understand instructional material better and are more prepared for class(Gurung, 2003; Narloch et al., 2006). Completing work outside of class may also contribute tostudent satisfaction and sense of self-determination (Sergis et al., 2018).
Completing work outside of class is a significant component of the flipped classroom model.
In a flipped classroom, “students gain first exposure to new material outside of class, usually
via reading or lecture videos, and then use class time to do the harder work of assimilating thatknowledge, perhaps through problem-solving, discussion, or debates” (Brame, 2013, p. 1). The
flipped model has been shown to improve learning performance (Bhagat et al., 2016), increase
student satisfaction (Bösner et al., 2015), engage students (Khanova et al., 2015), improvecritical thinking (van Vliet et al., 2015) and enhance application skills (Liou et al., 2016).
However, there are also challenges, including students not completing out-of-class work (Al‐Zahrani, 2015), and instructors’ inability to know if students have completed work (Fautch,
2015). Students commonly fail to spend adequate amounts of time studying outside of class(Akçayır & Akçayır, 2018; Lai & Hwang, 2016) and those students may not succeed in aflipped class (Sayeski et al., 2015). It has been suggested that engaging students during out-ofclass work and keeping them accountable are key factors for student success in the flippedmodel (Hwang et al., 2015).
While the flipped model’s effectiveness in increasing student outcomes has been wellestablished, less attention has been paid to its impact on students’ internal dispositions or
intrinsic motivation to remain engaged in the learning process (Sergis et al., 2018). A student’s
engagement and learning are influenced by their motivation (Elliot, 2019). Student motivation
to engage with the out-of-class content becomes critical in a flipped environment as students
who fail to do so are not able to fully participate in class (Aidinopoulou & Sampson, 2017) andincreasingly fall behind as a flipped course progresses (Chen et al., 2014).
Inspired by the possibilities, and aware of the challenges, the University of Hawaiʻi at Mānoa’s(UHM) Biology department partnered with the College of Education’s Distance Course Design
and Consulting Group (DCDC) to redesign its introductory Biology course, BIOL 171, toincorporate the flipped classroom model. Biology team members included the course professor
and lab coordinator serving as subject matter experts. DCDC team members included a project
manager, instructional designer, web developer, programmer, and graphic designer. From fall2017 to summer 2018, the OpenStax Biology 2e textbook was customized, then piloted in fall2018 with 463 students. This study examined students’ perceptions of using the customized
textbook. The questions guiding the study were:
1. How did BIOL 171 students perceive the design features added to the OpenStax
Biology 2e textbook?
2. Did the new design features influence students’ motivation and satisfaction whenparticipating in the course?
3. What improvements to the textbook did students suggest?

69
71

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Customizing the OpenStax Biology 2e Textbook
The OpenStax Biology 2e textbook is an Open Educational Resource (OER), as it resides in
the public domain and is released under an intellectual property license allowing for free useor re-purposing (Atkins et al., 2007). Educator reported benefits of OER include knowing thatall students have access to course materials (Weller et al., 2015), an increased sense of control
over curriculum, better ability to serve the needs of diverse learners, and positive changes inpedagogy (Pitt, 2015).
Students report favoring online OER textbooks over traditional print texts, finding them more
up-to-date and useful (Feldstein et al., 2012). Students have also reported satisfaction with thequality of OER materials (Gil et al., 2013; Pitt et al., 2013), and their accessibility (Weller etal., 2015). In addition, students also report that OER textbooks support their course work, andwould recommend OER to their classmates (Hilton et al., 2013). Another common reason forstudents preferring OER textbooks is low cost (Lindshield & Adhikari, 2013; Petrides et al.,2011). High prices for traditional textbooks have resulted in students choosing to not takecourses (Donaldson et al., 2012), not purchase materials while enrolled in a course (Allen, 2011)
and to be more concerned about book costs than tuition (Bonner, 2014 as cited in Pitt, 2015).
The textbook was redesigned to support a flipped classroom. A fundamental aspect of the
flipped model is students interacting with content out-of-class before engaging in learning
activities face-to face. This pre-class work provides students with a sense of ownership of theirlearning (O’Flaherty & Phillips, 2015) and has been attributed in part to the overall success ofthe flipped model (Gross, 2015). The textbook was specifically focused on to enhance students’
out-of-class experience and accountability.
Conceptual Framework
The redesign of the textbook was conceptually guided by Gagne’s Conditions of Learning andTheory of Instruction (1985) and Deci and Ryan’s (2012) Self-Determination Theory.
Considering the flipped model from an instructional design perspective, the primary functionof the textbook is to present content. However, according to the Conditions of Learning andTheory of Instruction (1985), presenting content is not enough. To effectively learn, students
must move through a series of learning events. Providing learner guidance and opportunities
for practice with feedback after content presentation are key learning events. In the absence ofsufficient learner guidance, practice, and feedback students in flipped learning environments
often lack the self-regulation to be successful with the out-of-class content (Sun et al., 2017).
A number of the features added to the textbook were designed to support Gagne’s learning
events relevant to out-of-class work including providing learner guidance, eliciting
performance and providing feedback on performance. To enhance learner guidance,
organization and navigational features were added. Additional chapters were added explaining
navigation of the textbook and providing tips on being a successful online learner. Text andfigure rollovers were included to reinforce concepts introduced. To provide opportunities forpractice, interactive flashcards and text and video quizzes were added. Immediate feedback
was provided and quizzes could be taken an unlimited number of times. Feedback on overall
progress was provided via the redesigned table of contents displaying the student’s status oneach of the quizzes.
The additional features of the redesigned textbook were also influenced by Self-Determination
Theory. Through the lens of Self-Determination Theory (Deci & Ryan, 2012), a learner’s
intrinsic motivation is strengthened when their actions feel autonomous and they feel

70
72

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

competent or able to master an activity. Intrinsically motivated learners tend to persist atlearning tasks, making them more likely to achieve (Vansteenkiste et al., 2006). Therefore,
learning materials used as a part of a flipped class should enhance perceptions of autonomy
and competence to foster motivation. A number of features were added to increase feelings ofautonomy. First, the textbook was designed to be easily accessed via mobile devices allowing
access from any location at any time. Second, quiz status icons were added to give students
continual feedback on their progress. Third, the table of contents was redesigned based on due
dates rather than topics. Each of these features was designed to enhance students’ sense of
autonomy as they completed out-of-class work. To increase students’ sense of competencewith the content, practice flashcards and text and video quizzes were added to each chapter.
Text and Video Quizzes
The number of undergraduate students who do not complete textbook readings or postpone
them until immediately before an exam is increasing (Clump et al., 2004; Johnson & Kiviniemi,
2009; Sappington et al., 2002). Failure to complete reading assignments is detrimental tolearning (Johnson & Kiviniemi, 2009), and negatively impacts in-class performance (Gurung,
2003; Narloch et al., 2006). Regularly scheduled quizzes have been associated with increased
exam scores and course performance (Johnson & Kiviniemi, 2009). Flipped students needformative assessment opportunities to understand what they know and do not know (Rotellar& Cain, 2016; Slomanson, 2014) and benefit from incentives to prepare for class (Kim et al.,2014). Additionally, flipped instructors need methods for assessing students’ understanding ofmaterial (Kim et al., 2014).
To enhance student performance, engagement and accountability, quizzes were embedded ineach chapter. A portion of final grades were based on overall quiz completion. Two types ofquizzes were added, text and video. Both types used a multiple-choice format and were createdusing the H5P plugin. Figure 1 shows a text quiz.

Figure 1: Screenshot of a text quiz
Eight additional videos were added to the text. Quizzes were embedded in these videos using
the H5P interactive video feature. As seen in Figure 2, quizzes paused the video with questions
for students to answer before they could continue watching.

71
73

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Figure 2: Screenshot of a video quiz
Glossary Term Practice Flashcards
Flashcards are known to increase vocabulary comprehension and retention (Fitzpatrick et al.,2008), are more likely to be used when provided by an instructor (Burgess & Murray, 2014),
and are most effective when used over time (Cepeda et al., 2006). The customized textbook
embedded interactive glossary term flashcards in each chapter using the original textbook’s
terms. Flashcards were built with Quizlet, a free, online, effective (Wright, 2016) flashcard
application. Flashcard use was optional, and not recorded.
Chapter Due Dates
Consistent pre-class assignments prepare students for a flipped class (Baepler et al., 2014) andcontribute to increased outcomes (Gross et al., 2015). Flipped classes should incorporate
preferred features of online classes including clear structure, easy navigation and schedulingmilestones (Bergstrand & Savage, 2013; Crews & Butterfield, 2014; Rabbany et al., 2011;
Wanner & Palmer, 2015). It is important for flipped students to know what they need to doprior to class (Rotellar & Cain, 2016), as self-regulation is often a challenge for students intechnology mediated environments (Shyr & Chen, 2018). This can be communicated to flippedclass students by providing a study schedule (Mason et al., 2013).
As displayed in Figure 3, the redesigned textbook used a calendar-like format that organizedchapters by their due dates rather than topic. This Due Dates page was shown upon logging
into the text.

72
74

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Figure 3: Chapters organized by due dates with quiz status icons
Each chapter was displayed as a single webpage. Groups of chapters due on the same date weredisplayed as circular icons at the top and bottom of each chapter page. As displayed in Figure
4, rolling over a chapter icon displayed the chapter’s title and quiz status. Clicking on a chapter
icon would take a student to that chapter.

Figure 4: Rollover feature displaying chapter title and individual student’s quiz status
Quiz Status Icons
Students are often not aware of how they are doing in a course until it is too late (Pistilli &Arnold, 2010). An online student dashboard “provides a visual display of the important
information needed to achieve one or more goals, consolidated and arranged on a single screen
so the information can be monitored at a glance” (Teasley, 2017, p. 378). Quiz status icons
were displayed with each chapter depicting student progress. As seen in Figures 3 and 4, eachquiz was represented by a circle. At a glance, a student could see their quiz status.
Figure and Text Rollovers
In the original textbook, when clicking on a figure reference would take readers back to theoriginal figure’s location. To reduce clicks and keep readers in their location, the redesigned

73
75

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

textbook added a rollover feature that popped a figure up in place any time it was referred toin the text. The figure rollover feature is seen in Figure 5.

Figure 5: Figure rollover feature
As seen in Figure 6, a glossary term in-text rollover feature was also added. Terms appeared
as underlined text, indicating that they could be rolled over to display their definition.

Figure 6: Text rollover feature
Chapter Feedback
At the end of every chapter, students were able to provide comments and ratings on a 1 to 5
“star” scale, see Figure 7. Feedback could be given a descriptive tag to identify it as acompliment, fix or recommendation. The feedback was then displayed to instructors in their
dashboard.

Figure 7: Student feedback feature

74
76

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Additional Chapters
A comprehensive course orientation may help reduce confusion for online students regarding
course layout and expectations (Morris & Finnegan, 2008). For flipped students it may
additionally reduce fear or resistance to a new method (Lee et al., 2017). To orient students
and provide tips on being a successful online learner, two additional chapters, Start Here andOnline Learning Success, were added. Start Here included a video detailing logging in, taking
quizzes and monitoring progress. Online Learning Success provided online learning tips.
MethodThe purpose of this study was to examine students’ perceptions of the customized textbook.
All methods and instruments were approved by the University’s institutional review board on
human subjects research. Data were collected using an anonymous, online survey consistingof 16 Likert scale, three open-ended, and three demographic questions. As the textbook’s
redesign had been conceptually guided Gagne’s Theory of Instruction and the SelfDetermination Theory, the survey asked students to rate the features added to provide learnerguidance, practice with content, and feedback as well as questions about satisfaction andmotivation with the text. The survey was distributed in week 14 of the 16-week fall 2018semester. Quantitative data were analyzed using SPSS 25.0. Qualitative data were quantified
for frequency of favored features and improvement suggestions.
Findings
Participants
The majority, 96.3% (n = 446) of the 463 fall 2018 BIOL 171 students invited to participate
completed the survey. Students ranged from 17 to 38 years old, 70% (n = 312) were female,
29% (n = 131) were male and 3 opted to not identify gender.
Students’ Perceptions of the Design Features
Text and video quizzes. Students were positive about the text and video quizzes. As
summarized in Table 1, the overall mean score (M) for quizzes was 3.8, above the midpoint on
a scale of 1 (strongly disagree) to 5 (strongly agree) with a standard deviation of .870.
Specifically, 59% found the quizzes engaging, 62.3% agreed that the questions in the videos
helped them learn the content in the video, and 68.6% agreed that the text quizzes reinforced
the chapter content. In addition, 16.6% listed the embedded quizzes as the feature they liked
best about the customized textbook. Student 327 explained, “the quizzes encouraged me toreview and study the information being taught.” Student 345 said, “the textbook had...video
quizzes so that I could be engaged in my homework.”

75
77

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 1: Text and video quizzes
Text and Video
1-2 (%)3 (%)
4-5 (%)
Quizzes (M =3.8, SD = .870)
1. The text and
10.7
30.3
59
video quizzes in
the online
textbook were
engaging.
2. The questions
11.5
26.2
62.3
embedded in thevideos helped
me learn thevideo content.
3. The text
8.1
23.3
68.6
quizzes
reinforced thechapter content.
(1-2: strongly disagree - disagree; 3: neutral; 4-5: agree - strongly agree)

M (SD)
3.72 (1.008)

3.78 (1.042)

3.94 (0.980)

The results also revealed students with neutral or negative perceptions toward the quizzes. As
seen in Table 1, 23.3% to 30.3% of the students indicated neutrality about the benefits of thetext and video quizzes to their learning, and 8.1% to 11.5% indicated negative perceptions of
the quizzes. Some students suggested adding more questions to the text and video quizzes. For
example, Student 372 wrote, “More content on the quizzes that better reflect exams questions
i.e actual questions from the exams.” Student 137 said, “the quizzes more in depth because
they were very easy and i feel quite a few of them didn’t cover main ideas of certain sections.”
Student 159 suggested fixing the questions saying, “Some of the quizzes had answers that did
not make sense.”
Glossary term practice flashcards. As displayed in Figure 8, slightly over half (52.9%) of
the students reported using the flashcards in most, or every chapter while 28.1% of the students
did not use them in any, or in very few chapters.

Figure 8. Reported frequency of use of glossary term practice flashcards.
Additional design features. The majority of students were positive about the additional design
features. The overall mean score for these 5 questions was 4.2 with an SD of .653. As shown

76
78

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

in Table 2, the quiz status icons (item 6) were rated most highly (approximately 90%) with amean value of 4.59 and an SD of .781. Organization of the textbook (item 5) was second, with
82.5% (M = 4.35, SD = .894). The remaining questions regarding redesign elements (rollovers,
chapter due dates, and feedback option) received a smaller number of positive responses. In
order of highest to lowest, 78.7% of the students liked the rollover feature (M = 4.27, SD =.880), 76.2% preferred the text organized by chapter due dates (M = 4.26, SD = 1.062) and thefeedback option scored lowest with 33.6% of the students neutral and 13.5% negative on thisitem (M = 3.64, SD = 1.112).
Table 2: Additional design features of the customized OpenStax Biology textbook
Additional
Design Features(M = 4.2, SD =.653)
5. Theorganization of
the online
textbook was
logical and easy
to follow.
6. The online
textbook quizstatus icons(checkmarks incircles
representing
each quiz in achapter) helpedme track myprogress.
7. I liked therollover featurethat allowed me
to see the figuresin a chapter.
8. Given the
choice, I would
rather have theonline textbook
chapters
organized bywhen chaptersneed to be readversus a
traditional tableof contents.
9. Having theoption to

1-2 (%)3 (%)

4-5 (%)

M (SD)

3.2

14.3

82.5

4.35 (0.894)

2.7

7.6

89.8

4.59 (0.781)

2.7

18.6

78.7

4.27 (0.880)

6.1

17.7

76.2

4.26 (1.062)

13.5

33.6

52.9

3.64 (1.112)

77
79

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

providefeedback at theend of each bookchapter made me
feel more
involved in mylearning process.
(1-2: strongly disagree - disagree; 3: neutral; 4-5: agree - strongly agree)
Additional chapters. The Start Here and Online Learning Success chapters added to the
textbook were rated positively by approximately 60% of students. The overall mean for thetwo chapters was 3.8 with an SD of 1. As displayed in Table 3, 63.2% of students agreed that
the Start Here chapter helped them navigate the textbook, and 61.2% found the Online Learning
Success chapter helpful.
Table 3: Additional chapters
Additional
1-2 (%)3 (%)
4-5 (%)
Chapters (M =3.8, SD = 1.000)
10. The Start
10.8
26
63.2
Here: Textbook
Orientation
chapter helped
navigate thetextbook.
11. I found the
11
27.8
61.2
information in
the Online
Learning
Success chapter
helpful andwould advocate
keeping it in the
online textbook.
(1-2: strongly disagree - disagree; 3: neutral; 4-5: agree - strongly agree)

M (SD)
3.84 (1.111)

3.78 (1.112)

Format. As displayed in Table 4, 90.8% of students reported that they preferred an online, freeversion of the text over a hard copy for purchase (M = 4.7, SD of .918). Fewer than 10% of
students indicated otherwise.
When asked specifically about the customized textbook (item 13), a smaller number of students(70.4%) indicated their preference for the online version (M = 4.09, SD = 1.253). More students(15.9%) indicated their neutrality on item 13 and 13.7% reported not preferring the online
textbook over a hard copy. Agreement dropped 20.4% between item 12 and 13 when students
were asked about a general preference for a free online textbook and this textbook in particular.

78
80

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 4: Free online vs hard copy format
Format (M =1-2 (%)3 (%)
4-5 (%)
4.4, SD = .918)
12. I preferred
3.6
5.7
90.8
having a free
online textbook
over a hard copy
book I have topurchase.
13. I preferred
13.7
15.9
70.4
having thisonline textbook
over a hard copy
textbook.
(1-2: strongly disagree - disagree; 3: neutral; 4-5: agree - strongly agree)

M (SD)
4.7 (0.805)

4.09 (1.253)

Favorite features. To the optional, open-ended question, “What did you like best about the
textbook?”, “No cost” and “Online” were the two most frequently given answers, mentioned
by 31.6% and 24.9% students respectively. Student 298 said “It was easy to access anywhere
I was because I found myself forgetting to do my reading but I could still do it in my sparetime when I was out just on my phone!” Student 430 commented:
Having easy access to this textbook was amazing. I could complete readings and hw on
the bus which helped me stay on top of my work. Also, the interface was intuitive andvery organized. Not having to pay and carry around a textbook was very nice. I wish
all my classes adopted this system. Having the hw integrated into the textbook mademy life much easier.
Student 250 reported:
I love this book! It is the best bio book I have ever read! I really like how easy it is tokeep track of assignments and to know if I turned them in or not. I also like how at thestart of every assignment, there was a learning objective center for me to know what tofocus on. Above all else, I LOVED the flashcards, quizzes and matching game. I wish
all my classes used books like this. BEST BOOK EVER!
In referring to no cost, student 345 said, “I like that this textbook was free especially because
biology textbooks are very expensive.” Student 154 stated that a no cost textbook “lowers thecost of going to school for the student” and Student 353 said “even students like me who
struggle financially could still have the opportunity to use it.”
Other favorite aspects cited included organization (21.7%) ease of access (10.1%), practice
flashcards (4.5%), chapter due dates (3.4%), glossary (2.9%), and chapter summaries (1.1%).
A few students, 2.7%, mentioned the search feature with Student 181 saying “There is also a“search” button, where I could look-up topics whenever I don’t know what chapter they belong
to.”

79
81

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Motivation and Satisfaction
As shown in Table 5, the mean value for the textbook motivating students to study was 3.38
(SD = 1.213), slightly above the midpoint.
Table 5: Influence of the textbook on student motivation
1-2 (%)

3
(%)
32.7

4-5 (%)

M (SD)

14. The online
21.7
45.6
textbook
motivated me to
study for thiscourse.
16. Other
6.7
11
82.3
courses should
adopt similar
onlinetextbooks.
(1-2: strongly disagree - disagree; 3: neutral; 4-5: agree - strongly agree)

3.38 (1.213)

4.27 (1.016)

Item 16 asked students to respond to the statement: “Other courses should adopt similar online
textbooks.” The majority of students (82.3%) agreed or strongly agreed with this statement (M= 4.27, SD = 1.016).
Suggested Improvements
Sixty three percent of students made suggestions for improvements, summarized in Figure 10.
Frequencies are based on the number of times a suggestion was made, one student often had
more than one suggestion.

Figure 10. Student suggestions for improvement.
Eight students recommended having more visuals and the same number suggested having a
traditional table of contents. Student 371 stated that “More pictures would be helpful.” Student

80
82

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

366 preferred to “have a table of contents available for the chapters for easy navigation while
studying, looking back over content, etc.” Six students made suggestions about the searchfeature.
Students also suggested four features to add to the textbook: 1) audio in addition to text (2students), 2) a PDF download for the textbook (7 students), 3) due date notifications sent via
email (8 students), and 4) adding a highlight/annotation feature (10 students).
Discussion and Recommendations
The elements added to the textbook intended to provide learner guidance, practice with content
and feedback on performance were all well received by students. The textbook appears to haveaddressed students’ need for autonomy and competence as evidenced by their overallsatisfaction with the book and their motivation to use it to study for the course.
The two most favored aspects of the text were no cost and the affordances of it being online.
Others have also found no cost and online to lead to student satisfaction with OER (Feldstein
et al., 2012; Hilton et al., 2013; Petrides et al., 2011). The majority, (90.8%) agreed with thestatement: “I preferred having a free online textbook over a hard copy book I have to purchase.”
This aligns with previous studies in which students reported preferring OER texts due to low
or no costs (Lindshield & Adhikari, 2013; Petrides et al., 2011). Students may not purchasecostly textbooks when enrolled in a course, despite decreased learning outcomes (Allen, 2011).
No cost textbooks address equitability and students may feel more competent when they canequally access learning materials. Online access may also have contributed to feelings ofautonomy given that students appreciated anywhere, any time access.
While using unmodified OER resources has proven successful, customized OER has also beenshown to increase student learning outcomes (Mathew & Kashyap, 2019). The redesigned
textbook customizations were cited as favorite aspects of the text. Students were positive whenrating features incorporated to enhance learner guidance including the chapters viewed by due
dates, glossary term practice flashcards and the new rollover feature. The majority of students(89.8%) felt the quiz status icons helped them monitor their progress. The ability to monitor
progress is an important part of self-regulated learning (Zimmerman, 2002) and students havebeen shown to be more self-regulated when they feel more autonomous in their learning(Sierens et al., 2009).
The text and video quizzes in particular were the third most favored aspect of the text. This isencouraging as ungraded test-enhanced learning has been shown to have learning benefits. In
particular, completing ungraded tests prior to an exam can increase active information retrieval
on exams (Karpicke & Roediger, 2008; McDaniel & Masson, 1985) which has been associatedwith higher final exam scores (Khanna, 2015).
Students also had suggestions for improving the quizzes. These included more quizzes, morequestions, and better alignment with the exams. That students highly rated the quizzes andwanted more may have been due to them supporting feelings of competence with the course
content, perhaps because quizzes have been found to be more beneficial to students in contentfocused courses versus skills-focused courses (Khanna & Cortese, 2016). Aligning quizzes
more closely to exams may also improve student confidence with the content.

81
83

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Requests for additional and more comprehensive quizzes suggests that students are willing toengage even further with the textbook. This is significant as out-of-class work is a key factor
in students’ success in flipped classes (Hwang et al., 2015) and students typically do not spend
enough time on out-of-class work (Akçayır & Akçayır, 2018; Lai & Hwang, 2016).
The two new chapters were less well received by students. Over a third of students, 36.8%,
disagreed or were neutral that the Start Here chapter was helpful. Additionally, 38.8% were
neutral or disagreed that the Online Learning Success chapter was helpful. Specific features ofthese chapters may have led to students’ attitudes toward them. First, neither were directly
related to the course content. Second, while encouraged, they were not required and did nothave quizzes that impacted students’ grades.
Student response to the chapter feedback feature was also mixed. Only half agreed that “Having
the option to provide feedback at the end of each book chapter made me feel more involved in
my learning process.” This may have been due to instructors not reading or responding tostudent feedback. It may also have been due to lack of student use of the feature. Despite theopportunity, students often do not provide feedback. Common reasons include skepticism
about the possibility for change or a lack of skill in how to provide meaningful feedback(Svinicki, 2001). If receiving feedback is a goal, it should be more thoroughly incorporated
into the teaching and learning process because “giving feedback is a skill that can be learned”(Svinicki, 2001, p. 1). The design team suggests that future instructors emphasize feedback’s
importance and respond to it more regularly. Enhanced communication with the instructor may
also increase students’ intrinsic motivation to use the textbook. When looked at through thelens of SDT, in addition to competence and autonomy, learners also have a need for relatedness
or the feeling of being connected to others (Deci & Ryan, 2012). Students that engage in moredialog with their instructor may feel more connected to them.
Other suggestions for improvement included the ability to download an offline copy of the text
to reduce eye strain and enable highlighting and annotation. This result is not surprising as thelack of annotation ability and eye strain associated online texts were complaints in a study
using the original OpenStax Biology textbook (Watson et al., 2017) and “computer vision
syndrome” (Akinbinu & Mashalla, 2014) is a known effect of increased screen time. In fact,
almost a third (29.6%) of the students in this study disagreed with or were neutral on thisstatement: “I preferred having this online textbook over a hard copy textbook.” The complaints
regarding eye strain and lack of ability to highlight and annotate may be associated with thisresult. In response, students are now provided with a PDF file of the textbook upon request but
are still encouraged to use the online version in order to have their quiz results recorded.
In conclusion, implementing the new features to the textbook enhanced students’ experienceand supported the efforts to incorporate the flipped learning model into the course. Students
appreciated the customized text, specifically cost reduction, online availability, and theembedded quizzes. However, the new features added complexity to the implementation
process, which in turn led to additional development time and increased need for technicalsupport. Institutions considering similar endeavors should be prepared to provide ongoingsupport to students and instructors, and to be responsive to technical and performance issues
that may arise.
Viewing future textbook developments through the lens of motivation theory may also prove
beneficial to retain focus on student engagement, especially if part of a flipped classroom
design. While this textbook development and study focused primarily on students’ need for82
84

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

competence and autonomy, future textbook developments might benefit from also
incorporating features that foster students’ need for relatedness. Features to enhance
relatedness might include increased student-to-student or student-to-instructor communication
directly in the textbook platform or leveraging the inherent social affordances of mobile
devices to encourage collaborative learning through student sharing, peer assessment, andintentional community building activities.
Limitations
The scope of this study presents limitations to transferability of the results. Only one textbook
in the specific field of Biology was redesigned and tested with students enrolled in a single
semester. Additional textbooks in various fields, tested with a larger pool of students would
result in more transferable results. In addition, a goal of the textbook was to motivate students
to engage in out-of-class content. However, only one survey question specifically asked if theredesigned textbook motivated them to study. Future research by the authors on redesigned
textbooks will address student motivation and satisfaction on a more comprehensive level.
Conclusion
In conclusion, implementing the new features to the textbook enhanced students’ experienceand supported the flipped learning model in the course. Students appreciated many aspects ofthe customized text, specifically cost reduction, online availability, and the embedded quizzes.
However, the new features added complexity to the implementation process, which in turn led
to additional development time and increased need for technical support. Institutions
considering similar endeavors should be prepared to provide ongoing support to students andinstructors, and to be responsive to technical and performance issues that may arise.
Viewing future textbook developments through the lens of motivation theory may also prove
beneficial to retain focus on student engagement, especially if part of a flipped classroom
design. While this textbook and study focused on students’ need for competence and autonomy,
future textbook developments might benefit from incorporating features that foster students’
need for relatedness. Features to enhance relatedness might include increased student-tostudent or student-to-instructor communication directly in the textbook or leveraging theinherent social affordances of mobile devices to encourage collaborative learning throughstudent sharing, peer assessment, and intentional community building activities.

83
85

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Akçayır, G., & Akçayır, M. (2018). The flipped classroom: A review of its advantages andchallenges. Computers & Education, 126, 334–345.
https://doi.org/10.1016/j.compedu.2018.07.021
Al‐Zahrani, A. M. (2015). From passive to active: The impact of the flipped classroom
through social learning platforms on higher education students’ creative thinking.
British Journal of Educational Technology, 46(6), 1133–1148.
https://doi.org/10.1111/bjet.12353
Baepler, P., Walker, J. D., & Driessen, M. (2014). It’s not about seat time: Blending,flipping, and efficiency in active learning classrooms. Computers & Education, 78,
227–236. https://doi.org/10.1016/j.compedu.2014.06.006
Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every classevery day. International Society for Technology in Education.
Bergstrand, K., & Savage, S. V. (2013). The chalkboard versus the avatar: Comparing theeffectiveness of online and in-class courses. Teaching Sociology, 41(3), 294–306.
https://doi.org/10.1177/0092055X13479949
Bhagat, K. K., Chang, C.-N., & Chang, C.-Y. (2016). The impact of the flipped classroom
on mathematics concept learning in high school. Journal of Educational Technology& Society, 19(3), 134–142. https://www.jstor.org/stable/jeductechsoci.19.3.134
Bösner, S., Pickert, J., & Stibane, T. (2015). Teaching differential diagnosis in primary careusing an inverted classroom approach: Student satisfaction and gain in skills andknowledge. BMC Medical Education, 15(1), 1–7.
https://doi.org/10.1186/s12909-015-0346-x
Brame, C. (2013, January 31). Flipping the classroom. Vanderbilt University website.
https://wp0.vanderbilt.edu/cft/guides-sub-pages/flipping-the-classroom/
Burgess, S. R., & Murray, A. B. (2014). Use of traditional and smartphone app flashcards in
an introductory psychology class. Journal of Instructional Pedagogies, 13, 1–7.
http://search.proquest.com/docview/1697499938
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practicein verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin,
132(3), 354–380. https://doi.org/10.1037/0033-2909.132.3.354
Chen, Y., Wang, Y., Kinshuk, & Chen, N.-S. (2014). Is FLIP enough? Or should we use the
FLIPPED model instead? Computers & Education, 79, 16–27.
https://doi.org/10.1016/j.compedu.2014.07.004
Clump, M., Bauer, H., & Bradley, C. (2004). The extent to which psychology students read
textbooks: A multiple class analysis of reading across the psychology curriculum.
Journal of Instructional Psychology, 3(31), 228–232.
Crews, T., & Butterfield, J. (2014). Data for flipped classroom design: Using student
feedback to identify the best components from online and face-to-face classes.
Higher Education Studies, 4(3), 38–47. https://doi.org/10.5539/hes.v4n3p38
Deci, E. L., & Ryan, R. M. (2012). Self-determination theory. In Handbook of Theories ofSocial Psychology (Vol. 1, pp. 416–436). Sage Publications Ltd.
https://doi.org/10.4135/9781446249215.n21

84
86

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Fautch, J. (2015). The flipped classroom for teaching organic chemistry in small classes: Is
it effective? Chemistry Education Research and Practice, 16(1), 179–186.
https://doi.org/10.1039/C4RP00230J
Feldstein, A., Martin, M., Hudson, A., Warren, K., Iii, J. H., & Wiley, D. (2012). Opentextbooks and increased student access and outcomes. European Journal of Open,Distance and E-Learning, 15(2), 1–9. http://search.proquest.com/docview/1314327088
Fitzpatrick, T., Al-Qarni, I., & Meara, P. (2008). Intensive vocabulary learning: A casestudy. The Language Learning Journal, 36(2), 239–248.
https://doi.org/10.1080/09571730802390759
Gagne, R. M. (1985). The conditions of learning and theory of instruction (4th ed.).
Wadsworth Publishing Company.
Gross, D., Pietri, E. S., Anderson, G., Moyano-Camihort, K., & Graham, M. J. (2015).
Increased preclass preparation underlies student outcome improvement in the flippedclassroom. CBE—Life Sciences Education, 14(4), 1–8.
https://doi.org/10.1187/cbe.15-02-0040
Gurung, R. A. R. (2003). Pedagogical aids and student performance. Teaching of
Psychology, 30(2), 92–95. https://doi.org/10.1207/S15328023TOP3002_01
Hilton, J., Gaudet, D., Clark, P., Robinson, J., & Wiley, D. (2013). The adoption of open
educational resources by one community college math department. The International
Review of Research in Open and Distributed Learning, 14(4), 37–50.
https://doi.org/10.19173/irrodl.v14i4.1523
Hwang, G.-J., Lai, C.-L., & Wang, S.-Y. (2015). Seamless flipped learning: A mobile
technology-enhanced flipped classroom with effective learning strategies. Journal ofComputers in Education, 2(4), 449–473. https://doi.org/10.1007/s40692-015-0043-0
Johnson, B. C., & Kiviniemi, M. T. (2009). The effect of online chapter quizzes on exam
performance in an undergraduate social psychology course. Teaching of Psychology,
36(1), 33–37. https://doi.org/10.1080/00986280802528972
Karpicke, J., & Roediger, H. (2008). The critical importance of retrieval for learning.
Science, 319(5865), 966–968. https://doi.org/10.1126/science.1152408
Khanna, M. (2015). Ungraded pop quizzes: Test-enhanced learning without all the anxiety.
The Teaching of Psychology, 42(2), 174–178.
https://doi.org/10.1177/0098628315573144
Khanna, M. M., & Cortese, M. J. (2016). The benefits of quizzing in content-focused versus
skills-focused courses. Scholarship of Teaching and Learning in Psychology, 2(1),
87–97. https://doi.org/10.1037/stl0000051
Khanova, J., Roth, M. T., Rodgers, J. E., & McLaughlin, J. E. (2015). Student experiences
across multiple flipped courses in a single curriculum. Medical Education, 49(10),1038–1048. https://doi.org/10.1111/medu.12807
Kim, M. K., Kim, S. M., Khera, O., & Getman, J. (2014). The experience of three flippedclassrooms in an urban university: An exploration of design principles. The Internetand Higher Education, 22, 37–50. https://doi.org/10.1016/j.iheduc.2014.04.003
Lai, C.-L., & Hwang, G.-J. (2016). A self-regulated flipped classroom approach to
improving students’ learning performance in a mathematics course. Computers &Education, 100, 126–140. https://doi.org/10.1016/j.compedu.2016.05.006

8587

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Lee, J., Lim, C., & Kim, H. (2017). Development of an instructional design model for
flipped learning in higher education. Educational Technology Research andDevelopment, 65(2), 427–453. https://doi.org/10.1007/s11423-016-9502-1
Lindshield, B. L., & Adhikari, K. (2013). Online and campus college students like using an
open educational resource instead of a traditional textbook. Journal of Online
Learning and Teaching, 9(1), 26–38. Retrieved from http://krex.kstate.edu/dspace/handle/2097/16660
Liou, W.-K., Bhagat, K. K., & Chang, C.-Y. (2016). Beyond the flipped classroom: A
highly interactive cloud-classroom (HIC) embedded into basic materials science
courses. Journal of Science Education and Technology, 25(3), 460–473.
https://doi.org/10.1007/s10956-016-9606-8
Mason, G. S., Shuman, T. R., & Cook, K. E. (2013). Comparing the effectiveness of an
inverted classroom to a traditional classroom in an upper-division engineeringcourse. IEEE Transactions on Education, 56(4), 430–435.
https://doi.org/10.1109/TE.2013.2249066
Mathew, S., & Kashyap, U. (2019). Impact of OER Materials on Students’ Academic
Performance in Undergraduate Astronomy Course. Journal of STEM Education:
Innovations and Research, 20(1), 46–49.
McDaniel, M. A., & Masson, M. E. (1985). Altering memory representations throughretrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition,
11(2), 371–385. https://doi.org/10.1037/0278-7393.11.2.371
Morris, L., & Finnegan, C. (2008). Best practices in predicting and encouraging student
persistence and achievement online. Journal of College Student Retention: Research,
Theory & Practice, 10(1), 55–64. https://doi.org/10.2190/CS.10.1.e
Narloch, R., Garbin, C. P., & Turnage, K. D. (2006). Benefits of prelecture quizzes.
Teaching of Psychology, 33(2), 109–112.
https://doi.org/10.1207/s15328023top3302_6
O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A
scoping review. The Internet and Higher Education, 25, 85–95.
https://doi.org/10.1016/j.iheduc.2015.02.002
Petrides, L., Jimes, C., Middleton‐Detzner, C., Walling, J., & Weiss, S. (2011). Opentextbook adoption and use: Implications for teachers and learners. Open Learning:
The Journal of Open, Distance and e-Learning, 26(1), 39–49.
https://doi.org/10.1080/02680513.2011.538563
Pistilli, M. D., & Arnold, K. E. (2010). Purdue signals: Mining real-time academic data toenhance student success. About Campus, 15(3), 22–24.
https://doi.org/10.1002/abc.20025
Porcaro, P. A., Jackson, D. E., McLaughlin, P. M., & O’Malley, C. J. (2016). Curriculum
design of a flipped classroom to enhance haematology learning. Journal of Science
Education and Technology, 25(3), 345–357. https://doi.org/10.1007/s10956-0159599-8
Rabbany, R., Takaffoli, M., & Zaïane, O. R. (2011). Analyzing participation of students inonline courses using social network analysis techniques. Proceedings of Educational
Data Mining, 21–30.

86
88

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Rotellar, C., & Cain, J. (2016). Research, perspectives, and recommendations on
implementing the flipped classroom. American Journal of Pharmaceutical
Education, 80(2), 1–9. https://doi.org/10.5688/ajpe80234
Ryan, T. E. (2006). Motivating novice students to read their textbooks. Journal of
Instructional Psychology, 33(2), 136–140. Retrieved fromhttp://eres.library.manoa.hawaii.edu/login?url=http://search.ebscohost.com/login.asp
x?direct=true&db=a9h&AN=21597697&site=ehost-live
Sappington, J., Kinsey, K., & Munsayac, K. (2002). Two Studies of Reading Compliance
Among College Students. Teaching of Psychology, 29(4), 272–274.
https://doi.org/10.1207/S15328023TOP2904_02
Sayeski, K. L., Hamilton-Jones, B., & Oh, S. (2015). The efficacy of IRIS STAR legacy
modules under different instructional conditions. Teacher Education and Special
Education, 38(4), 291–305. https://doi.org/10.1177/0888406415600770
Sergis, S., Sampson, D. G., & Pelliccione, L. (2018). Investigating the impact of flippedclassroom on students’ learning experiences: A self-determination theory approach.
Computers in Human Behavior, 78, 368–378. https://doi.org/10.1016/j.chb.2017.08.011
Shyr, W.-J., & Chen, C.-H. (2018). Designing a technology-enhanced flipped learning
system to facilitate students’ self-regulation and performance. Journal of Computer
Assisted Learning, 34(1), 53–62. https://doi.org/10.1111/jcal.12213
Sierens, E., Vansteenkiste, M., Goossens, L., Soenens, B., & Dochy, F. (2009). The
synergistic relationship of perceived autonomy support and structure in theprediction of self-regulated learning. British Journal of Educational Psychology,
79(1), 57–68. https://doi.org/10.1348/000709908X304398
Slomanson, W. R. (2014). Blended learning: A flipped classroom experiment. Journal of
Legal Education, 64(1), 93–102. Retrieved fromhttps://www.jstor.org/stable/24716075
Sun, J. C.-Y., Wu, Y.-T., & Lee, W.-I. (2017). The effect of the flipped classroom approach
to OpenCourseWare instruction on students’ self-regulation. British Journal of
Educational Technology, 48(3), 713–729. https://doi.org/10.1111/bjet.12444
Svinicki, M. D. (2001). Encouraging your students to give feedback. New Directions for
Teaching and Learning, 2001(87), 17–24. https://doi.org/10.1002/tl.24
Teasley, S. D. (2017). Student facing dashboards: One size fits all? Technology, Knowledge
and Learning, 22(3), 377–384. https://doi.org/10.1007/s10758-017-9314-3
van Vliet, E. A., Winnips, J. C., & Brouwer, N. (2015). Flipped-class pedagogy enhances
student metacognition and collaborative-learning strategies in higher education but
effect does not persist. CBE—Life Sciences Education, 14(3), 1–10.
https://doi.org/10.1187/cbe.14-09-0141
Vansteenkiste, M., Lens, W., & Deci, E. L. (2006). Intrinsic versus extrinsic goal contents in
self-determination theory: Another look at the quality of academic motivation.
Educational Psychologist, 41(1), 19–31. https://doi.org/10.1207/s15326985ep4101_4
Wanner, T., & Palmer, E. (2015). Personalising learning: Exploring student and teacher
perceptions about flexible learning and assessment in a flipped university course.
Computers & Education, 88, 354–369.
https://doi.org/10.1016/j.compedu.2015.07.008

8789

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Watson, C. E., Domizi, D. P., & Clouser, S. A. (2017). Student and faculty perceptions ofopenstax in high enrollment courses. The International Review of Research in Openand Distributed Learning, 18(5), 288–304.
https://doi.org/10.19173/irrodl.v18i5.2462
Wright, B. A. (2016). Transforming vocabulary learning with Quizlet. In P. Clements, A.
Krause, & H. Brown (Eds.) Transformation in Language Education. JALT.
Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into
Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2
Corresponding author: Ariana Eichelberger
Contact email: ariana@hawaii.edu

8890

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Learning Management System Adoption in Higher Education Using the Extended
Technology Acceptance Model
Marissa R. Fearnley
De La Salle-College of Saint Benilde Manila
Philippines
Johnny T. Amora
De La Salle-College of Saint Benilde Manila
Philippines

89
91

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
A learning management system is capable of enriching instruction and acceptance of thislearning technology by users is crucial to its successful application in higher education. This
study investigated factors that influenced adoption of a learning management system by higher
education teachers using the technology acceptance model which incorporates three external
constructs: system quality, perceived self-efficacy and facilitating conditions. Data collectedfrom faculty respondents (n=127) through an online survey were examined by employing
partial least squares-structural equation modeling. While several relationships were confirmed,
others were not supported by this research. Results showed that both system quality andperceived self-efficacy strongly influenced perceived usefulness, which in turn indirectly
affected attitudes towards the technology and behavioral intention. Additionally, system
quality directly affected perceived ease of use and attitudes toward technology use. The strong
and direct influence of perceived self-efficacy on perceived usefulness and perceived ease ofuse suggests that faculty with positive beliefs about their ability to use the learning management
system will regard it as both useful and easy to use. Facilitating conditions, on the other hand,affected neither perceived ease of use nor attitudes. Implications for practice, policy andpotential research directions are likewise presented.
Keywords: higher education, learning management system, technology acceptance model,
facilitating conditions, perceived self-efficacy, system quality

90
92

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Emerging technologies in education are driving colleges and universities to progressively
infuse their use in higher education (Rodriguez & Anicete, 2010; Alharbi & Drew, 2014;
Bermúdez-Hernández et al., 2017). Given their incontestable importance (Revythi & Tselios,2019), instructors are challenged to incorporate them as a means to complement conventional
learning environments (Lee et al., 2009; Rhode et al., 2017), enhance learners’ experiences,and improve academic outcomes (Parkman et al., 2018).
A learning management system (LMS) is a web-based application capable of transforming
face-to-face sessions by offering students a space for online learning (Wichadee, 2015). Usinga LMS is an effective way of delivering instruction to students by offering 24/7 access to course
content, while enabling convenient course creation and management for teachers (Bousbahi &Alrazgan, 2015). Despite the perceived benefits of using LMSs, many faculty members remain
hesitant to adopt them as a teaching tool (Wichadee, 2015; Zanjani et al., 2016). Moreover,teachers tend to underutilize this educational technology despite its widespread availability inhigher education settings (Fathema & Sutton, 2013; Bousbahi & Alrazgan, 2015). The
variables that affect faculty adoption of the technology include teachers’ perceptions, selfefficacy beliefs and instructional goals, as well as the availability of resources, support services,and time (Baturay et al., 2017; Siyam, 2019).
Teachers play an important role in carrying out any innovation in the classroom (Alharbi &Drew, 2014). This is why determining the variables that motivate teachers to provide atechnology-supported learning environment to their students is essential (Teo et al., 2012). This
paper aims to assess the factors that affect how teachers accept and adopt a learning
management system. Additionally, it seeks to confirm whether the technology acceptancemodel (TAM) in its extended version is applicable in a local higher education context.
Literature Review
Learning Management Systems
Learning management systems are platforms that offer a variety of integrated tools fordelivering and managing online instruction. Whether open source (e.g., Moodle, Sakai) orcommercial (e.g., Blackboard, Brightspace D2L), most LMSs are flexible, easy to use,
accessible and user-friendly (Kasim & Khalid, 2016).
With a LMS, an instructor can create online course content and subsequently manage thatcourse to enhance critical thinking abilities and promote collaboration among university
students (Zanjani et al., 2016). LMSs offer many tools such as online group chats, discussion
threads, video conferencing, lecture materials, learning modules, grading and course
evaluations, all of which may be customized to suit specific instructional needs (Fathema etal., 2015; Walker et al., 2016). According to Anshari et al. (2017), non-traditional forms oflearning supported by online approaches to instruction positively affect both teachers andlearners. Some of the benefits provided by a LMS include organized course content, enhanced
student engagement, improved autonomy among learners, convenient submission ofrequirements and immediate feedback (Adzharuddin et al., 2013; Cavus, 2015; Alenezi, 2018).
The utilization of LMSs to aid in educational initiatives has become widespread amongcolleges and universities over the years (Walker et al., 2016). Higher education institutions use
them to supplement face-to-face learning sessions, as well as support blended instruction anddistance learning (Klobas & McGill, 2010). Their significant contribution to instructionaldelivery notwithstanding, financial investment and technical demands are critical factors in the91
93

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

selection of a particular LMS (Kasim & Khalid, 2016). In addition, hindrances to LMS
utilization reported by teachers include: (1) deficient computer units and inadequate computercompetence; (2) problematic integration of technology into the curriculum; and (3) a lack ofsupervisory and technical personnel (Li, 2007; Baturay et al., 2017). Other concerns that
impede the delivery of technology-supported instruction in higher education are systeminfrastructure, effort from faculty, and satisfaction with the system (Surry et al., 2005).
Technology Acceptance Model
Before an innovation, such as a new LMS, can be implemented, it has to be accepted (Rogers,
2003). Investigating technology acceptance helps determine the purpose of teachers’
technology use (Scherer et al., 2019). Of the many theories and models that attempt to describe
how users accept and adopt a given technology (Ahmad et al., 2010; Shih-Chih et al., 2011),
one of the most prominent is the Technology Acceptance Model (TAM) (Park, 2009; Scherer
et al., 2019). Introduced by Davis (1989), the TAM has been implemented in different domains
of information technology and information systems (Shih-Chih et al., 2011), and has becomea useful model in exploring adoption behavior of a particular technology in various contexts(Fathema & Sutton, 2013).
The model (Figure 1) shows how perceived usefulness and perceived ease of use are strong
determinants of attitudes towards technology use (Siyam, 2019). Perceived usefulness
represents users’ views on the degree to which a technology facilitates improvement in job
performance, while perceived ease of use indicates users’ expectations that use of a technology
is easy and effortless (Davis, 1989; Lai & Savage, 2013). Both determinants influence users’
attitudes toward technology use (Baturay et al., 2017), and are themselves influenced byexternal variables (Shih-Chih et al., 2011). These external variables include subjective norm,
job relevance, output quality, and result demonstrability (Venkatesh & Davis, 2000), among
others. Users’ attitudes configure their behavioral intention, which in turn defines actualtechnology use (Alharbi & Drew, 2014; Revythi & Tselios, 2019).

Figure 1: Original technology acceptance model by Davis et al. (1989)
Over the years, different versions of the TAM have been employed in many empirical studies(Scherer et al., 2019). Other parameters that have been linked to the TAM are system quality,
computer self-efficacy and facilitating conditions (Fathema et al., 2015; Salloum et al., 2019),
as well as personal innovativeness (Mazman Akar, 2019) and access to technology andplanning time (Siyam, 2019).
The model proposed by Siyam (2019) consisted of the following external variables: job
relevance, access to technology, self-efficacy and planning time. Of the four factors studied,

92
94

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

only self-efficacy and planning time were found to be significant and positive determinants oftechnology acceptance among special education teachers. A related study conducted byParkman et al. (2018) in the United Arab Emirates sought to determine how 82 female preservice teachers accepted environments that were adequately supported by technology. This
investigation utilized the original TAM constructs and two external variables, namely
perceived user resources and computer self-efficacy, as predictors of behavioral intention.
Results showed that behavioral intention was strongly correlated with perceived usefulness,
perceived ease of use and computer self-efficacy. Between the two variables used to extend theTAM, computer self-efficacy was found to have a greater predictive power than perceived user
resources. In Turkey, 476 pre-service university teachers were surveyed by Baturay et al.
(2017) to investigate the relationship among competence in computer use, attitude towards
computer-assisted instruction and intention to use technology, and whether these external
constructs may be determined by four factors: gender; computer ownership; access to theInternet; and daily computer usage. Findings revealed that the correlations among three of theconstructs were statistically significant. Of the four factors examined, however, only daily
computer use was found to positively affect computer competence. The studies describedabove highlight the importance of competence in computer use or computer self-efficacy in the
adoption of a particular technology. In the context of this paper, the same construct is expressedas perceived self-efficacy.
In Serbia, knowledge of mathematics, subjective norm and facilitating conditions were usedby Teo and Milutinovic (2015) in their version of the TAM to assess factors that affecttechnology use in mathematics teaching among 313 pre-service teachers. Their findings
revealed that attitude towards technology use was the only variable with a direct effect on
behavioral intention. The TAM was also used by Fathema et al. (2015) to explore thedeterminants of technology acceptance among 560 university teachers and teaching assistantsin the United States whose LMS use was non-mandatory. Structural equation modelingrevealed that system quality, perceived self-efficacy and facilitating conditions were usefuldeterminants of attitudes toward use. Another study among 487 Turkish pre-service teachers
revealed that perceived usefulness, attitudes toward use and computer self-efficacy directly
affect behavioral intention.Perceived ease of use, facilitating conditions and technologicalcomplexity indirectly influence technology acceptance and perceived usefulness is a powerfuldeterminant of intention to use a particular technology (Teo et al., 2012).
While previous studies conducted by Teo and Milutinovic (2015), Fathema, et al. (2015), andTeo et al. (2012), incorporated facilitating conditions as an external variable of the TAM, only
the work of Fathema et al. (2015) explained how system quality affects technology acceptance.
A more recent study by Salloum et al. (2019) revealed that among university students, system
quality is a significant predictor of perceived ease of use but not perceived usefulness. Thus,
the inclusion of system quality as an external variable was deemed an important factor in thecontext of using a commercial LMS.
Research Framework and Hypotheses
The current study adopted the extended technology acceptance model proposed by Fathema etal. (2015) which integrated system quality, perceived self-efficacy, and facilitating conditions
as factors that influence adoption and implementation of a LMS by faculty. Additionally, theeffect of system quality on perceived ease of use was included in this investigation. A total of
fourteen (14) hypotheses were formulated to test this study’s model (Figure 2).

93
95

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Figure 2: Extended technology acceptance model for learning management systems by
Fathema et al. (2015)
System quality (SQ) characterizes desirability of the system (i.e., the LMS) in terms offunctions, speed, features, content, and interaction capability (Delone & Mclean, 2003;
Fathema et al., 2015). If the LMS is desirable, teachers will find it useful, view it positively,
and use it intentionally. Another significant factor in adopting an information system isperceived self-efficacy (SE), or self-assessment of one’s capability to carry out the tasks neededto accomplish a set of outcomes (Bandura, 1993; Gong et al., 2004; Nikou & Economides,2019). In this study, SE indicates how confident a faculty member is in his or her ability tooperate, navigate, and work with the LMS. Lastly, facilitating conditions (FC) are factors thatcan positively or negatively affect how easy or difficult it is for an individual to accomplish aparticular job (Teo, 2010). In terms of LMS use, FC is measured by the availability of technicalsupport, capacity building opportunities, and related resources. If FC are present, faculty
members are expected to show high perceived ease of use and positive attitudes towards LMSuse.
The first eight (8) hypotheses address the effects of the external variables SQ, SE and FC on
the original TAM constructs. These are:
H1: System quality directly affects PU.
H2: System quality directly affects PE.
H3: System quality directly affects AT.
H4: System quality directly affects BI.
H5: Perceived self-efficacy directly affects PU.
H6: Perceived self-efficacy directly affects PE.
H7: Facilitating conditions directly affect PE.
H8: Facilitating conditions directly affect AT.
The other six (6) hypotheses examine the relationships among the original TAM variables
namely, PE, PU, AT and BI.

94
96

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

H9: Perceived ease of use directly affects PU.
H10: Perceived ease of use directly affects AT.
H11: Perceived usefulness directly affects AT.
H12: Perceived usefulness directly affects BI.
H13: Attitude toward LMS use directly affects BI.
H14: Behavioral intention directly affects AU.
As educational technologies are being integrated into teaching and learning practices, theexamination of factors that are vital to user adoption becomes increasingly important (Rhodeet al., 2017). This investigation is aimed at confirming the applicability of the extended TAM(Fathema et al., 2015), with a focus on the effects of system quality, perceived self-efficacy
and facilitating conditions.
Methods
Participants of the Study
This research explored the acceptance and adoption of Brightspace, a learning management
system being utilized by faculty members at a private college in Manila, Philippines.
Brightspace is a commercial LMS designed by the Desire2Learn (D2L) Corporation (Moseley& Ajani, 2015). It is a cloud-based platform with a suite of tools for delivering flexible
instruction and facilitating engaged learning.
A total of 127 out of 250 (51%) served as the study sample. These teachers had experience
using the LMS for at least one academic term (i.e., 14 weeks) for purposes beyond downloadingclass lists and uploading course syllabi. Hair et al. (2012) recommended that the minimum
sample for a partial least squares-path model be equal to the number of structural paths for theconstruct with the greatest complexity multiplied by 10, which, in this case, would require aminimum sample of 40 participants. Thus, the study’s actual sample of 127, which is more than
three times the minimum requirement, is deemed adequate for the purposes of thisinvestigation. Table 1 presents demographic data related to the sample group.
The respondents in this study comprised of 51% male and 49% female teachers who use some,or all, of the available features of the LMS. A majority (75%) of the respondents were parttime faculty, and 98% (125 out of 127) were current users of the LMS at the time the data werecollected. Three of the respondents were teachers with administrative positions (i.e., otherfaculty) who were given teaching loads. A great majority (73%) of the faculty had completed
graduate education, and about 44% (56 out of 127) had a minimum of four years of teachingexperience in the institution.

95
97

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 1: Demographic profile of respondents (n = 127)
Variables
Sex
FemaleMale
Education
BA/BS
Doctorate/Post-doctorate
Master’s
Classification
Full-Time Teaching FacultyPart-Time Teaching Faculty
Other Faculty
Current LMS users
Yes
No
No. of years of teaching in the institution
Below 55 to 910 to 14
15 to 19
20 & above
Total

n

%
62
65

48.82
51.18

34
18
75

26.77
14.17
59.06

29
95
3

22.83
74.80
2.36

125
2

98.43
1.57

56
24
24
19
4
127

44.09
18.90
18.90
14.96
3.15
100.00Research Instrument
The research instrument used for data collection was an online survey that consisted of threeparts: the first part described the objectives of the study and sought informed consent from therespondents; the second part asked about the respondents’ gender, education, facultyclassification, current LMS use, and number of years of teaching at the institution; the thirdpart consisted of survey items related to LMS adoption. The TAM-related survey items wereadopted from Fathema et al. (2015) and were re-worded to align with the context of thisresearch. A total of 28 statements were used to assess the eight constructs: SQ (4 items), SE (3items), FC (3 items), PE (4 items), PU (4 items), AT (4 items), BI (3 items), and AU (3 items).
All items were measured on a four-point Likert scale with 1 = strongly disagree, 2 = disagree,
3 = agree, and 4 = strongly agree.
Data Gathering Procedure
A web-based survey created using Google Forms was sent to faculty members through theirofficial email addresses in the College. Data collection was done in the middle of the academic
year, approximately one year after the college shifted to Brightspace from an open source LMS.
Conducting this study was supported and monitored by the institutional research office.
Compliance with prescribed ethical guidelines was ensured through informed consent from therespondents by requiring them to signify understanding of the nature and purposes of the study,offering no rewards for participation, and providing an option to be excluded from the study incase they change their mind. Personal identifiable information of the respondents (i.e., email
address) were accessible only to the researcher who analyzed the data.

96
98

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Data Analysis
This is a quantitative study that applied partial least squares-structural equation modeling (PLSSEM) to determine the association among eight constructs related to TAM (PU, PE, AT, BI,
AU, SQ, FC, and SE). This technique enables estimation of complex models with several
constructs (Revythi & Tselios, 2019) but requires less stringent assumptions on sample size,
distribution and normality (Hair et al., 2019).
Findings
This section follows the stages of PLS-SEM model analysis and interpretation suggested by
Hulland (1999). The first part presents the results generated from the tests of reliability andvalidity, and the second part shows the tests of hypotheses related to the structural model.
Reliability and Validity
Construct reliability evaluates the degree of consistency between the reflective item and theintended measure (Roldan & Sanchez-Franco, 2012; Kock, 2015; Amora et al., 2016).
Construct reliability is deemed adequate when composite reliability (CR) and Cronbach’s alpha(CA) are at least 0.70 (Fornell & Larcker, 1981; Nunnaly & Bernstein, 1994; Kock, 2015).
Convergent validity assesses whether the level of understanding by the respondents on theitems associated with each variable matches the intention of the designer of the instrument. In
PLS-SEM, there are two methods used to test convergent validity. First, item loadings related
to the variable should be at least 0.50 in value, and statistically significant (p < 0.5). Itemloading indicates the correlation between the item and the variable. Next, average varianceextracted (AVE) should be at least 0.50 (Hair et al., 2012; Kock, 2015; Amora et al., 2016).
AVE measures the overall dispersion attributed to the construct against that of observational
error (Fornell & Larcker, 1981).
The values in Table 2 indicate that all eight constructs were above the recommended 0.70
guideline for construct validity. Likewise, convergent validity of the constructs in the structuralmodel were deemed sufficient with item loadings and AVE values recorded well above 0.50.
Table 2: Item loadings, AVE, and reliability of the constructs
Constructs

No. of
items
3

Item
loadings***.77-.96

AVE

CA

CR

.78

.86

.92

2. Behavioral Intention (BI)3.91-94

.85

.91

.95

3. Perceived Usefulness (PU)

4.91-.95

.88

.95

.97

4. Perceived Ease of Use (PE)4.81-.91

.73

.88

.92

5. Attitude Towards Using (AT)4.87-.91

.81

.92

.95

6. Facilitating Conditions (FC)3.86-.92

.81

.88

.93

7. Perceived Self-Efficacy (SE)3.90-.94

.85

.91

.95

8. System Quality (SQ)

4.73-.85

.68

.84

.90

1. Actual Use (AU)***p < .001.

97
99

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Discriminant validity is present when respondents do not confuse the items of a particularvariable with those of other variables in the instrument, particularly in terms of meaning (Kock,2015). If the √AVE of a variable (i.e., any of the diagonal values) is greater than the coefficients(i.e., off-diagonal values) for any combination of this variable with another, then the items inthat variable have a strong correlation (Teo, 2010). In this case, all diagonal values are higher
than the related off-diagonal values which shows acceptable discriminant validity for all
constructs (see Table 3).
Table 3: Correlation coefficients and AVE of the constructs
Constructs

1

2

3

4

5

6

7

1. Actual Use (AU)0.89

2. Behavioral Intention (BI)

0.62

0.92

3. Perceived Usefulness (PU)

0.60

0.79

0.94

4. Perceived Ease of Use (PE)

0.49

0.64

0.72

0.86

5. Attitude Towards Using (AT)

0.61

0.87

0.84

0.72

0.90

6. Facilitating Conditions (FC)

0.30

0.55

0.57

0.65

0.56

0.90

7. Perceived Self-Efficacy (SE)

0.55

0.67

0.68

0.81

0.72

0.64

0.92

8. System Quality (SQ)

0.39

0.66

0.72

0.79

0.74

0.69

0.73

8

0.83

Note. Diagonal values represent the square root of AVE of constructs (√AVE), while the offdiagonal values are the correlation among the constructs.
Tests of convergent validity, discriminant validity and reliability satisfied the conditions for
estimating the structural model: average path coefficient (APC) = 0.304 (p < .001), average Rsquared (ARS) = 0.656, p < .001; average adjusted R-squared (AARS) = 0.649, p < .001;
average block VIF (AVIF) = 3.073 (acceptable if ≤5, ideally ≤ 3.3); average full collinearity
VIF (AFVIF) = 3.773 (acceptable if ≤ 5, ideally ≤ 3.3); and Tenenhaus goodness of fit (GoF)= 0.726 (large ≥ 0.36) (Kock, 2015). Overall, the fit and quality indices of the structural modelin the present study fell within acceptable limits.
Test of Structural Model
Table 4 presents the parameter estimates and outcomes for proposed relationships in theextended TAM. Of the fourteen (14) hypotheses, only 10 were supported by the data.

100
98

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 4: Parameter estimates of the extended technology acceptance model for LMS
HypothesesPathβp-valuef2
Results
H1
SQ àPU
.340
.000
.082
Supported**
H2
SQ à PE
.390
.000
.081
Supported**
H3
SQ à AT
.237
.003
.084
Supported*
H4
SQ à BI
.016
.428
.088
Not supported
H5SE à PU
.226
.004
.159
Supported*
H6
SE à PE
.470
.000
.380
Supported**
H7
FC à PE
.081
.177
.053
Not supported
H8FC à AT
.005
.476
.003
Not supported
H9PE à PU
.278
.000
.205
Supported**
H10
PEà AT
.108
.108
.080
Not supported
H11
PU à AT
.590
.000
.499
Supported**
H12
PU à BI
.219
.005
.173
Supported*
H13
AT à BI
.666
.000
.574
Supported**
H14
BI à AU
.633
.000
.401
Supported**2
Note. f is Cohen’s effect size: .02=small, .15=moderate, .35=large; β=path coefficient*p < .05, **p < .01
In terms of system quality, results of the structural model revealed that SQ directly affected
PU, PE, and AT. Positive β values (i.e., path coefficients) indicated that high SQ led to highPU, PE and AT. Additionally, SQ significantly affected BI. For perceived self-efficacy, it canbe noted that SE significantly affected PU and PE. Moreover, the extent of the effect of SE canbe described as large on perceived ease of use (f2 = .380) and moderate on perceived usefulness(f2 = .159). FC did not have a significant effect on PE and AT.
PE had a positive and significant effect on PU, but no statistically significant effect on AT.
The positive β value of PE on PU suggested that when teachers reported ease of LMS usage,
they also perceived the LMS to be useful. Moreover, PU had a more dominant effect on AT (f2= .499) than BI (f2 = .173). The data also offered evidence that faculty who found the LMS
useful would have positive attitudes and strong intentions to use the LMS.
AT positively and significantly affected BI, and in turn, BI directly affected AU. This meansthat teachers with favorable attitudes toward LMS usage were likely to have stronger
behavioral intentions which led to actual technology use. Similar to the effect of AT on BI (f2= .574), behavioral intention influenced AU to a large extent (f2 = .401).
Discussion
This research sought to confirm the applicability of the extended TAM (Fathema et al., 2015)
in a local higher education context. While it confirmed several relationships from the model,
four hypotheses were not supported by this research. More specifically, the effects of SQ on
BI, FC on PE and AT, and PE on AT were all not statistically significant. Findings from thisstudy expand the existing body of knowledge on TAM by reinforcing a new dimension to therelationships that exist between SQ and PE.

101
99

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Effects of External Variables
System quality is a significant external variable that affects users’ attitudes towards technology
usage (Fathema & Sutton, 2013). Aligned with the work of Fathema et al. (2015) and Salloum
et al. (2019), this study revealed that SQ positively and significantly affects PU and PE. Thismeans that a high-performing LMS in terms of quality will be perceived by teachers as both
useful and easy to use. Consistent with the finding of Fathema et al. (2015), SQ has a significant
effect on AT but not BI. This means that the characteristics of a LMS, although desirable, do
not assure its consequent utilization by faculty.
Teachers’ perceived self-efficacy has a significant influence on PU and PE, which confirms
previous findings that self-efficacy is a critical factor in assessing whether technology use will
be successful or not (Ahmad et al., 2010; Teo et al., 2012; Fathema et al., 2015). This result
suggests that teachers with high SE also have high levels of comfort and confidence that using
the LMS will help them achieve their goals at work. Based on previous research, high SE
among teachers translates to improved perceptions and stronger intentions to use the LMS(Baturay et al., 2017; Parkman et al., 2018).
The provision of technical support for technology use is rated by faculty as an enabler (Siyam,2019), and its absence is a barrier to technology integration in the classroom (Lim & Khine,2006). According to Fathema et al. (2015), teachers are likely to develop more approving
attitudes toward the technology when facilitating conditions (i.e., training, tutorials, support,etc.) are present. In terms of FC, the results of this study were not in agreement with earlier
work by Teo (2010) and Fathema et al. (2015) which reported FC to have a significant effecton PE. It could be possible that adequate facilitating conditions were not fully in place at thetime of data collection, which made the effects of FC on PE weak and insignificant.
Alternatively, SQ and SE may be quite high such that teachers do not feel the need for FC, andthe lack of facilitating conditions presented no significant effect on PE. Consistent with one of
the findings of this research, however, are related studies that found no significant association
between FC and AT (Teo et al., 2012; Teo & Milutinovic, 2015).
Effects of Original TAM Constructs
The findings of this research are generally consistent with previous TAM-related work thatreported strong relationships between PE and PU, and PU and AT (Teo, 2010; Teo et al., 2012;
Fathema et al., 2015, Teo & Milutinovic, 2015; Stockless, 2018; Siyam, 2019). Between PU
and PE, PU is a stronger predictor of AT compared to PE. Contrary to the researchers’
expectation, this investigation showed that PE does not significantly predict AT. This suggeststhat faculty who perceive the LMS to be easy to use do not necessarily possess positive attitudes
towards the LMS. This may be explained by the high perceived self-efficacy reported byteachers which could strongly affect how easy it is for them to work with the LMS. Whenteachers feel confident in their abilities to use a technology and find no difficulty in its
utilization, their attitudes towards it may likely be unaffected. Thus, PE had no significant
effect on AT. As noted earlier, SE has the largest influence on PE compared to SQ and FC,
thus, lending further support to the above explanation. The positive effect of AT on behavioral
intention, and the similar effect of BI on actual use are all in agreement with previous work by
Fathema et al. (2015), Teo (2010), and Ahmad et al. (2010).
Conclusion and Recommendation
The technology acceptance model has been modified numerous times based on emerging
technologies (Benbasat & Barki, 2007), and these various modifications may provide some

100
102

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

explanation as to why there is no single model that is unanimously accepted by the researchcommunity (Stockless, 2018). This may be attributed to differences in educational context,
characteristics of respondents, and emphasis as regards to the external variables utilized in theinvestigation (Wu & Liu, 2015). While the study results provide an acceptable level of support
for the TAM and its extended version, the relationships among constructs that were notconfirmed by this research (i.e., the effects of SQ on BI, FC on PE and AT, and PE on AT)
merit further examination.
Among the three external variables included in this investigation, perceived self-efficacy has
the strongest influence on PU and PE. As such, there is further evidence that teachers withpositive beliefs about their capacity for technology will find a LMS both useful and easy touse. This research also confirms the desirability of the learning management system being
utilized in the college, because system quality positively and significantly affects PU, PE andAT.
The current study provides an empirical validation of the extended TAM proposed by Fathema
et al. (2015). It offers practical implications in terms of practice, policy and future related work.
For practitioners, self-confidence in their ability (i.e., perceived self-efficacy) to use atechnology is an essential element of acceptance and adoption. Moreover, faculty with high SE
may be recruited as partners in promoting LMS use or engaged as mentors for other teachers
who are at the early stages of LMS integration. For policy makers who make decisions on the
procurement of educational technologies by their respective institutions, system quality should
be considered as an important selection criterion. By assessing system quality, administrators
can prioritize the kind of institutional support to be put in place for users of the technology(Rhode et al., 2017). For researchers, succeeding studies on the TAM may seek to further
explore the effects of other external variables including personal innovativeness andtechnology access (Mazman Akar, 2019), job relevance and planning time (Siyam, 2019),
management support (Ayele & Birhanie, 2018), subjective norm (Teo, 2010) and computer
anxiety (Venkatesh & Bala, 2008), among others. While the single method of data collectionused in this study limits generalizing the findings to other populations, this research provides
a better understanding of how higher education teachers accept, adopt, and use technology in
the Philippine context. Replication studies may also consider a mixed-method approach togenerate qualitative data that can enrich the explanatory power of the different TAM constructs.

101
103

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Adzharuddin, N.A., & Ling, L. H. Ling. (2013). Learning management systems (LMS)
among university students: Does it work? International Journal of e-Education, eBusiness, e-Management and e-Learning, 3(3), 248–252.
https://doi.org/10.7763/IJEEEE.2013.V3.233
Ahmad, T., Madarsha, K., Zainuddin, A., Ismail, N., & Nordin, M. (2010). Faculty’s
acceptance of computer based technology: Cross-validation of an extended model.
Australasian Journal of Educational Technology, 26(2), 268–279.
https://doi.org/10.14742/ajet.1095
Alenezi, A. (2018). Barriers to participation in learning management systems in Saudi
Arabian universities. Hindawi Education Research International.
https://doi.org/10.1155/2018/9085914
Alharbi S., & Drew, S. (2014). Using the technology acceptance model in understanding
academics’ behavioural intention to use learning management systems. International
Journal of Advanced Computer Science and Applications, 5(1), 143–155.
https://doi.org/10.14569/IJACSA.2014.050120
Amora, J., Ochoco, M. S., & Anicete, R. C. (2016). Student engagement and college
experience as mediators of the relationship between institutional support andacademic performance. Digital Journal of Lasallian Research, 12, 15–30.
Anshari, A., Almunawar, M. N., Shahrill, M., Wicaksono, D. K., & Huda, M. (2017).
Smartphones usage in the classrooms: Learning aid or interference? Education andInformation Technologies, 22(6), 3063–3079.
https://doi.org/10.1007/s10639-017-9572-7
Ayele, A. A., & Birhanie, B. K. (2018). Acceptance and use of e-learning systems: the caseof teachers in technology institutes of Ethiopian universities. Applied Informatics,
5(1). https://doi.org/10.1186/s40535-018-0048-7
Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning.
Educational Psychologist, 28(2), 117–148.
https://doi.org/10.1207/s15326985ep2802_3
Baturay, M. H., Gökçearslan, Ş., & Ke, F. (2017). The relationship among pre-service
teachers’ computer competence, attitude towards computer-assisted education, andintention of technology acceptance. International Journal of Technology Enhanced
Learning, 9(1), 1–13. https://doi.org/10.1504/IJTEL.2017.084084
Benbasat, I., & Barki, H. (2007). Quo vadis TAM? Journal of the Association for Information
Systems, 8(4), 211–218. https://doi.org/10.17705/1jais.00126
Bermúdez -Hernández, J., Chalela, S., Valencia Arias, J., & Valencia Arias, A. (2017).
Research trends in the study of ICT based learning communities: A bibliometricanalysis. EURASIA Journal of Mathematics Science and Technology Education,
13(5), 1539–1562. https://doi.org/10.1515/mjss-2017-0016
Bousbahi, F., & Alrazgan, M. S. (2015). Investigating IT faculty resistance to learning
management system adoption using latent variables in an acceptance technology
model. The Scientific World Journal, 2015, 1–11.
https://doi.org/10.1155/2015/375651

102
104

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Cavus, N. (2015). Distance learning and learning management systems. Procedia - Social
and Behavioral Sciences, 191, 872–877. https://doi.org/10.1016/j.sbspro.2015.04.611
Davis, F. (1989). Perceived usefulness, perceived ease of use and user acceptance of
information technology. MIS Quarterly, 13(3), 319–340.
https://doi.org/10.2307/249008
Delone, W. H., & Mclean, E. R. (2003). The Delone and Mclean information system success:A ten-year update. Journal of Management Information Systems, 19(4), 30–36.
https://doi.org/10.1080/07421222.2003.11045748
Fathema, N., Shannon, D., & Ross, M. (2015). Expanding the technology acceptance model(TAM) to examine faculty use of learning management systems (LMSs) in higher
education institutions. MERLOT Journal of Online Learning and Teaching, 11(2),210–232.
Fathema, N., & Sutton, K. L. (2013). Factors influencing faculty members’ learning
management systems adoption behavior: An analysis using the technology acceptancemodel. International Journal of Trends in Economics Management & Technology,
2(6), 20–28.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models withunobservable variables and measurement error. Journal of Marketing Research, 48,
39–50.
Groves, M. M., & Zemel, P. C. (2000). Instructional technology adoption in higher
education: An action research case study. International Journal of Instructional
Media, 27(1), 57–65.
Hair, J., Risher, J., Sarstedt, M., & Ringle, C. (2019). When to use and how to report theresults of PLS-SEM, European Business Review, 31(1), 2–24.
https://doi.org/10.1108/EBR-11-2018-0203
Hair, J., Sarstedt, M., Ringle, C., & Mena, J. A. (2012). As assessment of the use of partial
least squares structural equation modeling in marketing research. Journal of the
Academy of Marketing Science, 40, 414–433.
https://doi.org/10.1007/s11747-011-0261-6
Hulland, J. (1999). Use of partial least squares (PLS) in strategic management research: A
review of four recent studies. Strategic Management Journal, 20(2), 195–204.
https://doi.org/10.1002/(SICI)1097-0266(199902)20:2<195::AID-SMJ13>3.0.CO;2-7
Kasim, N. M. M., & Khalid, F. (2016). Choosing the right learning management system(LMS) for higher education institution context: A systematic review. International
Journal of Emerging Technologies in Learning, 11(6), 55–61.
https://doi.org/10.3991/ijet.v11i06.5644
Klobas, J. E. & McGill, T. J. (2010). The role of involvement in learning management system
success. Journal of Computing in Higher Education, 22(2), 114–134.
http://doi.org/10.1007/s12528-010-9032-5
Kock, N. (2015). WarpPLS 5.0 user manual. Script Warp Systems.
Lai, A., & Savage, P. (2013). Learning management systems and principles of good teaching:
Instructor and student perspectives. Canadian Journal of Learning & Technology,
39(3), 1–21. https://doi.org/10.21432/T24S39

103
105

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Lee, B. C., Yoon, J. O., & Lee, I. (2009). Learners’ acceptance of e-learning in South Korea:
Theories and results. Computers & Education, 53(4), 1320–1329.
https://doi.org/10.1016/j.compedu.2009.06.014
Li, Q. (2007). Student and teacher views about technology: A tale of two cities. Journal of
Research on Technology in Education, 39(4), 377–397.
https://doi.org/10.1080/15391523.2007.10782488
Lim, C. P., & Khine, M. S. (2006). Managing teachers’ barriers to ICT integration in
Singapore schools. Journal of Technology and Teacher Education, 14(1), 97–125.
Mazman Akar, S. G. (2019). Does it matter being innovative: Teachers’ technology
acceptance. Education and Information Technologies, 24, 3415–3432.
https://doi.org/10.1007/s10639-019-09933-z
Moseley, S., & Ajani, T. (2015). Users’ perceptions on the Brightspace learning management
system. Issues in Information Systems, 16(4), 10–19.
Nikou, S. A., & Economides, A. A. (2019). Factors that influence behavioral intention to use
mobile-based assessment: A STEM teachers’ perspective. British Journal of
Educational Technology, 50(2), 587–600. https://doi.org/10.1111/bjet.12609
Nunnaly, J. C., & Bernstein, I. H. (1994). Psychometric theory. Mc-Graw Hill.
Park, S. Y. (2009). An analysis of the technology acceptance model in understanding
university students’ behavioral intention to use e-Learning. Journal of Educational
Technology & Society, 12(3), 150–162.
Parkman, S., Litz, D., & Gromik, N. (2018). Examining pre-service teachers’ acceptance oftechnology-rich learning environments: A case study. Education and Information
Technologies, 23, 1253–1275. https://doi.org/10.1007/s10639-017-9665-3
Revythi, A., & Tselios, N. (2019). Extension of technology acceptance model by using
system usability scale to assess behavioral intention to use e-learning. Education andInformation Technologies, 24(4), 2342–2355. https://doi:10.1007/s10639-019-09869-4
Rhode, J., Richter, S., Gowen, P., Miller, T., & Wills, C. (2017). Understanding faculty useof the learning management system. Online Learning, 21(3), 68–86.
https://doi:10.24059/olj.v%vi%i.1217
Rodriguez, M., & Anicete, R. (2010). Students’ views of a mixed hybrid Ecology course.
MERLOT Journal of Online Learning and Teaching, 6(4), 791–798.
Rogers, E.M. (2003). Diffusion of innovations (5th ed.). Free Press.
Roldan, J. L., & Sanchez-Franco, M. J. (2012). Variance-based structural equation modeling:
Guidelines for using partial least squares in information systems research. In M.
Mora, O. Gelman, A. Steencamp, & M. Raisinghani (Eds.), Research methodologies
in engineering of software systems and information systems: Philosophies, methods
and innovations (pp. 193–221). Information Science Reference.
Salloum, S. A, Alhamad, A. Q. M., Al-Emran, M., Monem, A. A., & Shaalan, K. (2019).
Exploring students’ acceptance of e-learning through the development of acomprehensive technology acceptance model. IEEE Access, 7, 128445–128462.
https://doi.org/10.1109/ACCESS.2019.2939467

104
106

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Scherer, R., Siddiq, F., & Tondeur, J. (2019). The technology acceptance model (TAM): A
meta-analytic structural equation modeling approach to explaining teachers’ adoptionof digital technology in education. Computers & Education, 128, 13–35.
https://doi.org/10.1016/j.compedu.2018.09.009
Shih-Chih, C., Shing-Han, L., & Chien-Yi, L. (2011). Recent related research in technology
acceptance model: A literature review. Australian Journal of Business & Management
Research, 1(9), 124–127.
Siyam, N. (2019). Factors impacting special education teachers’ acceptance and actual use oftechnology. Education and Information Technologies, 24(3), 2035–2057.
https://doi.org/10.1007/s10639-018-09859-y
Stockless, A. (2018). Acceptance of learning management system: The case of secondary
school teachers. Education and Information Technologies, 23(3), 1101–1121.
https://doi.org/10.1007/s10639-017-9654-6
Surry, D. W., Ensminger, D. C., and Haab, M. (2005). A model for integrating instructional
technology into higher education. British Journal of Educational Technology, 36(2),327–329. https://doi.org/10.1111/j.1467-8535.2005.00461.x
Teo, T. (2010). Examining the influence of subjective norm and facilitating conditions on theintention to use technology among pre-service teachers: A structural equation
modeling of an extended technology acceptance model. Asia Pacific Education
Review, 11(2), 253–262. https://doi.org/10.1080/10494821003714632
Teo, T., & Milutinovic, V. (2015). Modelling the intention to use technology for teachingmathematics among pre-service teachers in Serbia. Australasian Journal of
Educational Technology, 31(4), 363–380.
https://doi.org/10.14742/ajet.1668
Teo, T. & Ursavas, O., & Bahcekapili, E. (2012). An assessment of pre-service teachers’
technology acceptance in Turkey: A structural equation modeling approach. The AsiaPacific Education Researcher, 21(1), 191–202.
Wu, C., & Liu, C. F. (2015). Acceptance of ICT-mediated teaching/learning systems forelementary school teachers: Moderating effect of cognitive styles. Education andInformation Technologies, 20(2), 381–401.
https://doi.org/10.1007/s10639-013-9290-8
Venkatesh, V., & Bala, H. (2008). Technology Acceptance Model 3 and a research agenda oninterventions. Journal of Information Technology, 39, 273–315.
https://doi.org/10.1111/j.1540-5915.2008.00192.x
Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptancemodel: Four longitudinal field studies. Management Science, 46(2), 186–204.
https://doi.org/10.1287/mnsc.46.2.186.11926
Walker, D., Lindner, J., Murphrey, T., & Dooley, K. (2016). Learning management system
usage: Perspectives from university instructors. Quarterly Review of Distance
Education, 17(2), 41–50.
Wichadee, S. (2105). Factors related to faculty members’ attitude and adoption of a learning
management system. The Turkish Online Journal of Educational Technology, 14(4), 53–61.

105
107

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Zanjani, N., Edwards, S., Nykvist, S., & Geva, S. (2016). LMS acceptance: The instructorrole. The Asia-Pacific Education Researcher, 25(4), 519–526.
https://doi.org/10.1007/s40299-016-0277-2
Corresponding author: Marissa R. Fearnley
Contact email: marissa.fearnley@benilde.edu.ph

106
108

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Digital Skills at Divine Word University, Papua New Guinea
Iwona Kolodziejczyk
Divine Word University
Papua New Guinea
Philip Gibbs
Divine Word University
Papua New Guinea
Cecilia Nembou
Divine Word University
Papua New Guinea
Maria Rodina Sagrista
Divine Word University
Papua New Guinea

107
109

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
The purpose of the study was to investigate the level of digital skills within a group of
university students in Papua New Guinea and their ability to meaningfully engage within thedigital world. The study also aimed to explore whether the traditionally recognized digital
divide continues between genders and place of origin, and between years of study and faculties.
This study utilizes the framework defined by Van Deursen and Van Dijk who identify
operational, formal, information and strategic skills. The study measured the internet skills ofstudents by asking 289 participants to complete sixteen assignments on the internet. The results
indicate that, on average, the tasks were completed as follows: 38% of operational skills, 13%
of formal internet skills, 30% of information skills and 28% of strategic internet skills. Ageand gender were not significant; however, performances were significantly different for
students from the capital city as compared to those coming from a rural town or village
environment, and performance improved as students moved through the four years ofuniversity training. Differences in groups from specific faculties are significant but require
further study to explain. If operational and formal skills are a necessary (but not sufficient)condition for performance of higher-level information and strategic skills, then tertiary
institutions, particularly those facing the effects of the digital divide, will need to ensure thatthose necessary skills are provided for. Having ensured digital competency at that level, further
efforts can be made to develop information and strategic skills to ensure a meaningful andcreative use of digital technologies.
Keywords: digital skills, digital literacy framework, university, developing country, Papua
New Guinea

108
110

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

The internet provides an endless source of information and entertainment, and also has thepotential to empower individuals to realize their educational and professional development.
However, just having access to Information Communication Technologies (ICTs) does notmean people are making use of them. Many people have access to these technologies, but theylack the skills to use them properly or the motivation to engage with them. Realising theimportance of the internet involves not simply monitoring the number of citizens subscribed to
an internet plan, but also assessing the patterns behind usage and skills such as production of
meaningful content, and engagement with technology or the benefits obtained from it (Fuchs& Horak, 2008; Godhe, 2019; Spante et al., 2018). Hence, digital skills are not simply
functional technological skills, but also the ability to find and select digital information, andthe knowledge essential for people to become “critical consumers” (Ameen & Gorman, 2009,
p. 99) of that information.
Research for this paper began as a study of the so-called “digital divide” whereby newtechnologies open a gap between those who can access these technologies and those who
cannot. Societal digital divides are those between citizens with and without formal education
and between rural and urban populations in their relation to technology. However, physicalaccess alone, without proper skills and adequate policies, is not enough to achieve the full
benefits of ICTs (Acilar, 2011; Chetty, Qigui, et al., 2018; Halford & Savage, 2010; Jin &Cheong, 2008). For this reason, nowadays many researchers expand the initial area of research,based on access and hardware, and introduce the so-called “second divide”, referring to thedifferent level of skills that allow users not only to access technology, but to make a fruitfuland effective use of it (Chetty, Qigui, et al., 2018; Lesame, 2013; Selvyn, 2004; van Deursen
et al., 2014).
The second divide, goes hand in hand with the term “digital literacy”, a new indispensable setof skills that allows users to take profit of and perform efficiently in digital environments(Eshet-Alkalai, 2004). It can be said that digital literacy is “as important as reading and writing
were in earlier centuries” (Katz, 2007, p. 4). In 2009, the ICT Development Index wasestablished with the intention that it become a tool to measure the level of ICT adoption andadvancement of a country and to contribute to adequate policy-making (Ayanso et al., 2011).
However, it has not been used uniformly and, for the purpose of our study, it does not provide
data for the country of Papua New Guinea. In order to start developing a base of reliable dataon the specific characteristics of the digital literacy in Papua New Guinea, our team decided toundertake a practical study to assess the digital skills of university students at the Madang
campus of Divine Word University.
Measuring Digital Skills
Given the importance of the digital skills for success in higher education as well as future
participation in the digitized world, substantial efforts have been made by researchers todevelop methods of measuring what has been termed “digital literacy”. Researchers approach
the issue from various perspectives. Reynolds (2016) observes that digital literacy often relates
to the extent of technology usage and thus measuring of digital literacy focuses on practicalskills. However, with the fast-changing technology, as well as with more exposure of the young
generation to technology and its practical application in everyday activities, measuring digitalskills has evolved in new directions to capture more complex functions and uses of technology.
The changing landscape of research done on digital skills is presented in Table 1, which gives
the example of four different approaches to measuring digital skills.

109
111

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 1: Four approaches to measuring digital skills
Four steps in digital
inclusions(Livingstone &Helsper, 2007, p. 10)

Six-skills of digital
literacy model(Eshet-Alkalai, 2004,2012)

Four types of digital
literacy skills (vanDeursen & van
Dijk, 2008, p. 4)

Digital competencies
framework (AlvarezFlores, et al., 2017, p.
545)

Information seeking:using the internet for
information finding
purposes

Photo-visual digitalskill: ability to use
graphical userinterface

Operational skills –the skills to operate
digital media

Games and emails:
using the internet for
information,entertainment andcommunication

Reproduction digitalskill – ability to use,
edit and manipulatedifferent forms of files
to produce new ones

Formal skills – theskills to handle thestructures of digital
media

Instant messaging anddownloading music:
using the internet forthe purposes describedin steps 1 and 2, andexpanding use forpeer-to-peer
engagement
Using the internet for
all previous steps
activities and for awide range ofinteractive and creativeuses.

Branching digital skill– ability to navigate
through hyperspace inboth non-orderly andnon-linear ways.

Information skills –the skills to locate
information in digital
media

Information –competency inbrowsing and filtering
data, evaluating andstoring data andfeedback
Communication –competency ininteraction and sharing
through digitaltechnologies,
participation andcollaboration throughdigital media andmanaging digital
identities
Content creation –competency indeveloping, integrating
and re-elaborating
digital content,
copyrights and licenses
and in programming

Information digitalskill – ability tocritically evaluate
online information

Strategic skills – theskills to employ the
information
contained in digital
media towards
personal (andprofessional)
development

Socio-emotional
digital skill - ability tocommunicate
effectively to createand share knowledge
onlineSafety – competency isprotecting devises anddata, protecting health,well-being andenvironment

Problem solving –competency in solving
technical problems,
creatively using digitaltechnologies andidentifying digitalcompetence needs

Real-time digital skill(added by Eshet in2012) – ability toeffectively execute
different tasks
simultaneously

110
112

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Although authors identified different stages in the process of engagement with technology,
they all point out the gradual evolution of the skills from the simple use of graphical userinterface and searching for information, to critical analysis of the information found, to more
complex tasks of using this information in order to create new digital artefacts and share themwith other online users.
While researching the digital divide among young people, Livingston and Helsper (2007)
identified four steps in digital inclusions which led them to categorizing young online users
into four categories: “basic users”, “moderate users”, “broad users” and “all-rounders” (p. 10).
These categories of users reflect a staged process of going online: from simple information
seeking (basic users), to using the internet for information, entertainment, and communication,
to expanding online activities to peer-to-peer engagement. The final stage includes a broad
variety of interactive and creative uses. The authors support the widespread assumption that“basic use makes for a narrow, unadventurous, even frustrating use of the internet, while moresophisticated use permits a broad-ranging and confident use of the internet that embraces newopportunities and meets individual and social goals” (p. 14).
Based on the notion that engagement with technology involves more than mere ability to use adigital device, Eshet-Alkalai (2004) proposes to evaluate digital skills through five-skills of adigital literacy model. To function effectively in the digital world, he observes, it requires a“variety of complex cognitive, motor, sociological, and emotional skills” (p. 93). Thus, heproposes a five-skills digital literacy framework, later (in 2012) extended into a six-skills
literacy framework. The proposed list of skills (photo-visual; reproduction; branching;
information; socio-emotional; and real-time) are claimed to be “survival skills” in a digitalized
world (Eshet-Alkalai, 2012, pp. 268–272) including in the context of today’s higher education(Komlayut & Srivatanakul, 2017).
In relation to measuring digital skills, Ilomäki, et al. (2016) observe that in policy related
papers, the term “skills” has been replaced by the term “competencies” (p. 655). Various papers
make an attempt to propose the digital competencies that are required from those who enterand function in the digital world (Gašová et al., 2018; Iordache et al., 2017). Included in Table
1 is a digital competencies framework developed by (Álvarez-Flores et al., 2017).
Van Deursen and Van Dijk (2008) observed that the models and framework available tomeasure digital skills were more on a conceptual level, which may contribute to a lack ofconsistency among researchers in their approaches to measuring digital skills. Thus, the authors
have made an attempt to move from the conceptual to the operational level. In their descriptionof the four types of digital skills they provided “operational definitions for operational, formal,
information and strategic skills, the measurements, and sample procedures” (p. 5). For each of
the type of skills, Van Deursen and Van Dijk provide measurement indicators. The first three
types of skills are related to an effective use of the internet. The indicators for operational skills
include skills in operating an internet browser, operating an online search engine andcompleting online forms. To measure formal skills, the indicators include skills in navigating
the internet through hyperlinks, and maintaining a sense of location while navigating theinternet. The information skills are measured by locating required information by choosing anappropriate search system, or website, ability to define search queries, selecting informationfrom search results, and evaluating information for accuracy of data and the reliability of thesources. The strategic skills are related to the purpose of the internet use and therefore involvethe skills to use the networks’ sources to achieve a specific goal as well as improve one’s status
in the society. To measure the strategic skills, the set indicators are: Taking advantage of the

111
113

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

internet by setting a particular goal, making the decision and taking the appropriate action toachieve the goal, and gaining the benefits belonging to this goal (pp. 5–9).
The framework used by Van Deursen and Van Dijk was to measure the internet skills of Dutch
citizens. The results show that although the Dutch population has a high level of operationaland formal internet skills, the level of information and strategic skills is much lower. The
authors concluded that surveys measuring only operational and formal skills may not reflect
actual digital skills of the researched population.
Our study adopted Van Deursen and Van Dijk’s framework. The main factors influencing ourselection of the framework were the specificity and detailed indicators developed for each of
the type of digital skills, and the practicality of the proposed research setup.
Digital Skills in the Papua New Guinea Context
For decades, the Papua New Guinea (PNG) Government has understood that command andcontrol of ICT is one of the keys to successful participation in the global digitalized world.
Thus, to ensure that the entire nation benefits from access to services provided through ICT,the government established a Department of Information and Communication in mid-1992.
Since then, although a number of policies and regulations were put in place, many attempts toimplement them were unsuccessful. Various barriers were identified, including the ad hoc wayof mounting projects resulting in incompatible systems being installed (Vaa, 2003), thediversity of the cultural and linguistic landscape, and the difficult geographical terrain (Henao,
2004). However, the government remains committed to strengthening the capacity andaccessibility of ICT across the entire country (Department of National Planning andMonitoring, 2010). In its vision, the government expects the higher education sector to be able
to prepare a skilful workforce prepared for the 21st-century work market. In line with the PNG
Development Strategic Plan 2010-2030, the Department of Education (2016) plans to provide
ICT training for all school teachers and for teaching ICT skills in primary and secondary
schools. The new curriculum for upper secondary schools (Department of Education, 2008)
introduces a Computer Studies program. In addition, the Department envisions establishingand maintaining successful e-learning infrastructure. However, despite ambitious plans, amajority of the student population, especially in rural areas, has no or little access to ICT(Department of Education, 2019; Leh & Kennedy, 2004; Ravinder, 2011; Trucano, 2014).
The higher education sector in PNG has a longer history of technology presence. Already in
1990, when there were only 250 internet users in the country, the internet was accessed at the
University of Technology in Lae (Kolodziejczyk, 2012). Since then, the application of ICT inhigher education institutions varied with generally much higher exposure to technology atuniversities than at colleges. However, as found by Kolodziejczyk, staff and students at all
institutions of higher education recognize technology as an important factor contributing tosuccess in teaching and learning as well as for a future professional career. In the same line,
the recent assessment of the PNG graduate labour market (Salonda et al., 2017) shows that
digital competency is recognized as one of the most important skills of university graduates.
Understanding the importance of digital skills for successful learning at the tertiary level (Gazi,
2016; Jiménez-Cortés et al., 2017; Komlayut & Srivatanakul, 2017) as well as the demand for
graduates equipped with digital skills (Chetty, Aneja, et al., 2018; Gašová et al., 2018; Ilomäki
et al., 2016), this research focuses on measuring the current digital skills of students at one of
the universities in PNG. Better understanding of the level of student digital competency may

112
114

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

advise teaching and learning strategies towards strengthening and developing skills that will
prepare students to function in digitalized environments.
Research Design
The research was inspired by the study of Van Deursen and Van Dijk (2008) on internet skills
and the digital divide of the Dutch population. The objective was to investigate the level ofdigital skills within a group of university students in PNG and their ability to meaningfully
engage within the online world. The study also aimed to explore whether the traditionallyrecognized digital divide continues between genders and place of origin, and between years ofstudy and faculties. For these reasons, the study was guided by two main research questions:
Q1: What are the levels of operational, formal, information and strategic digital skills of
undergraduate students at the Madang Campus of Divine Word University?
Q2: Are there statistically significant differences among the categories of gender, age,faculty of study, year of study, and residency (capital city, town or village)?
For comparability reasons, our study employed a similar approach to Van Deursen and Van
Dijk and used the quantitative method. Prior to the completion of internet-based assignments,
the participants were required to fill in a short questionnaire which collected sociodemographic information such as gender, age, faculty, year of study, and former place ofresidence before joining the university. Residence was considered significant because PNG has
only one city, the capital Port Moresby, with a population approaching half a million. There
are provincial towns, varying in size. However, the majority of the population live in small
villages, many of which are isolated, without electricity or modern amenities.
The tasks to be completed on the internet were prepared following the description of four typesof digital skills as defined in the original study:••••

Operational skills – the skills to operate digital media
Formal skills – the skills to handle the structures of digital media
Information skills – the skills to locate information in digital media
Strategic skills – the skills to employ the information contained in digital media
towards personal (and professional) development (van Deursen & van Dijk, 2008, p.
4).

A total number of 16 assignments were devised to measure various digital skills. Time wasallocated to complete each of the assessments. It required a total of 88 minutes to complete alltasks. All activities done on the internet were screen recorded for coding and analysis using
Active Presenter software.
For the purpose of the study a special computer lab was set up with 30 brand new laptops withthe same setup to ensure that all students were given the same tools. All computers wereconnected to the university’s network. To complete their tasks, students could choose from thethree popular web browsers (Mozilla Firefox, Internet Explorer and Chrome). After each groupcompleted the session, temporary files, browser history, favourites, cookies, and computerstorage and passwords were deleted to ensure that the next group work was not affected byprevious participants’ actions.

113
115

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

In line with the quantitative nature of the study, appropriate sampling was used. To achieve a
representative sample of the student population, the researchers randomly selected 12 groups
from among three faculties. For randomization, all group names were written on a piece of
paper and a lottery done in each of the faculties at different levels: year one, two, three, andfour. Two departments from within the Faculty of Business Studies and Informatics were
excluded from the selection process, Information Systems and Maths and Computing, as theirextensive experience with technology compared to others may have affected the results. All
selected groups were approached and invited to take part in the study. All were informed about
the voluntary nature of participation; a few students declined participation and left the lab
during a session. A total of 289 students took part in the study which is considered an adequate
number for a population of approximately 1400 to ensure valid results for quantitative analysis.1
More detailed socio-demographic information about participants is presented in Table 2.
Table 2: Socio-demographic profile of the study participants
GenderFaculty
Year of study

Place of origin

FemaleMale
Arts and Social Sciences
Business and Informatics
Health and Medical Sciences
First year
Second year
Third year
Fourth Year
Capital city
Town
Village159 (55%)
129 (45%)125 (43%)
66 (23%)
97 (34%)73 (25%)78 (27%)86 (30%)51 (18%)67 (23%)158 (55%)
63 (22%)The study obtained ethical clearance on April 4, 2017 from the Faculty of Arts and Social
Sciences Ethics Committee (ID# FASS/FS/3/2017).
Study results
As indicated earlier, the purpose of the study was twofold: to create a profile of digital skills
level among tertiary students and at the same time to explore possible digital divides along fivevariables: gender, age, faculty, year of study, and residency. After the general overview ofcompletion of tasks in each of the digital skills category, the following sections present moredetailed analysis of each of the four categories measured.
General Overview
Different types of digital skills were measured in four categories: Operational skills (9 tasks
completed); Formal skills (2 assignments); Information skills (3 assignments); and Strategic
skills (2 assignments). The completion of tasks was recorded as: 1 = not completed; 2= partiallycompleted; 3= completed. As expected, the highest percentage of task completion was in thelowest category – operational skills (38%). However, the most difficult for students to complete
were tasks in the formal skills category with only 13% of all tasks completed (Table 3).
1

We note the issues with the internet connection in the time students were involved in completing research tasks
and periodic unavailability of one of the government websites. In such cases, the affected results were notincluded in statistical analysis.

114
116

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 3: General summary of completion of tasks in four categories of digital skills
Completion of tasks
Operational tasks (9)Formal tasks (2)Information tasks (3)Strategic tasks (2)Mean (M)Standard
Deviation (SD)% of tasks completed

2.04
1.64
1.68
1.64

.814
.661
.816
.886

38
13
30
28

Operational Tasks
The results of a linear regression test indicate that faculty and year of study are the main
predictors of the level of operational skills.
Table 4: Linear regression of results of the number of operational tasks completed

GenderAge
Faculty
Year of study
Place of residency
R2
F

Beta
.087
-.088
.136
.386
-.075

Number of tasks completedt
1.562-1.5372.441
6.783-1.341.154
11.449

Sig.
.120
.126
.015
.000
.181
p<.001The ANOVA test2 shows that among three faculties, students from the Faculty of Medical
Sciences score highest in completion of operational tasks (F(2)=4.66, p=.01).
Table 5: ANOVA results for the operational skills category as completed by students in threefaculties

Arts and Social Sciences
Business and Informatics
Health and Medical
Sciences

Number of tasks completedM
SD% of all tasks
3.33
2.74
37
3.10
2.87
34
4.22
2.18
47

When analysed by year of study, the ANOVA results (Table 6) show that students in the fourthyear score highest on completion of operational skills with the lowest results for students in thefirst year of study (F(3)=15.391, p<.001).

2

ANOVA test (or analysis of variance test) is used to decide whether the differences between means ofobservations is due to chance or whether there are systematic effects that have caused scores of observations in
one or more groups to be statistically significantly different.

115
117

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 6: ANOVA results for number of operational skills tasks completed by students at
different levels of university study

First Year
Second Year
Third Year
Fourth Year

Number of tasks completedM
SD% of all tasks
2.34
2.40
26
2.96
2.69
33
4.52
2.27
50
4.67
2.45
52

Formal Skills
The results of linear regression test indicate that three main predictors of the level of formal
skills among participating students are year of study, place of residence and faculty.
Table 7: Linear regression of results of the number of formal tasks completed

GenderAge
Faculty
Year of study
Place ofresidency
R2
F

Number of tasks completed
Beta
t
-.038
-.663
-.080
-1.380-.163-2.861
.239
4.110
-.187
-3.275
.133
8.642

Sig.
.508
.169
.005
.000
.001

.000

The ANOVA test shows that among students at different years of study, students in the fourthyear score the highest in completion of formal skills (F(3)=7.905, p<.001). The significanceeffect is caused by the group of first year students whose results significantly differ from theother three groups (Table 8).
Table 8: ANOVA results for number of formal skills tasks completed by students at different
levels of university study

First Year
Second Year
Third Year
Fourth Year

Number of tasks completedM
SD% of all tasks
.05
.23
2.5
.35
.53
17.5
.27
.45
13.5
.39
.49
19.5

The ANOVA test shows that among students who came from a different residential background(lived either in capital city, town or village prior to their study at university), students who
lived in the capital city score the highest in completion of formal skills (F(2)=8,614, p<.001).
The significance effect is caused by the group of students who lived in the village whose results
significantly differ from the other two groups (Table 9).

116
118

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 9: ANOVA results for number of formal skills tasks completed by students comingfrom different residential backgrounds

Capital city
Town
Village

Number of tasks completedM
SD% of all tasks
.45
.530
22.5
.23
.436
11.5
.14
.353
7

The ANOVA test (Table 10) shows that among students in different faculties, students fromthe Faculty of Business and Informatics score significantly higher in completion of formal
skills that from other faculties (F(2)=14.038, p<.001). The significantly lower scores wererecorded among students in the Faculty of Health and Medical Sciences.
Table 10: ANOVA results for number of formal skills tasks completed by students indifferent faculties
Number of tasks completed
M
SD
Arts and Social Sciences
Business and Informatics
Health and Medical Sciences

.26
.47
.10

.442
.561
.306

% of all
tasks
13
23.5
5.1

Information Skills
The results of linear regression test (Table 11) indicate that the two main predictors of the levelof information skills among participating students are year of study and place of residence.
Table 11: Linear regression of results of the number of information tasks completed

GenderAge
Faculty
Year of study
Place of residency
R2
F

Number of tasks completed
Beta
t
-.017
-.299
-.044
-.734
-.016
-.275
.182
3.036
-.193
-3.258
.075
4.564

Sig.
.765
.463
.783
.003
.001
.001

The ANOVA test (Table 12) shows that scores of information tasks completion differ
significantly (F(3)=4.727, p<.003) among students at different levels of study with students
from first year scoring significantly lower than other groups.

117
119

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 12: ANOVA results for number of information skills tasks completed by students at
different levels of university study
Number of tasks completedM
SD% of all tasks
.59
.683
19.7
.97
.772
32.3
1.0
.826
33.3
.98
.836
32.7

First Year
Second Year
Third Year
Fourth Year

The ANOVA test shows that among students who came from different residential backgrounds(lived either in the capital city, town, or village prior to their study at university), students who
lived in the capital city score the highest in completion of information skills (F(2)=7.277,
p<.001). This effect is caused by the group of students who lived in the village whose results
significantly differ from the other two groups (Table 13).
Table 13: ANOVA results for number of information skills tasks completed by students fromdifferent residential backgrounds

Capital city
Town
Village

1.0597.9430.5714

Number of tasks completedM
SD% of all tasks.7361035.3.8235731.3.6889519

Strategic Skills
The results of linear regression test (Table 14) indicate that faculty is the only predictor of thelevel of strategic skills among participating students.
Table 14: Linear regression of results of the number of strategic tasks completed

GenderAge
Faculty
Year of study
Place ofresidency
R2
F

Number of tasks completed
Beta
t
-.107
-1.814
-.042
-.687
-.135
-2.266
.069
1.135
-.109
-1.820
.057
3.404

Sig.
.071
.492
.024
.257
.070

.005

The ANOVA test (Table 15) shows that students in the Faculty of Business and Informatics
scored significantly higher in completion of strategic skills than from other faculties(F(2)=9.234, p<.001). This effect is caused by the group of students in the Faculty of Healthand Medical Sciences whose results significantly differ from the other two groups.

118
120

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 15: ANOVA results for number of strategic skills tasks completed by students indifferent faculties

Arts and Social Sciences
Business and Informatics
Health and Medical
Sciences

Number of tasks completedM
SD% of all tasks
.57
.639
28.5
.80
.706
40
.38
.509
19
Discussion

The study objective was twofold; firstly, to investigate the levels of operational, formal,
information and strategic digital skills of undergraduate students at the Divine Word University
in Madang Campus; secondly, to explore any statistically significant differences among thecategories of gender, age, faculty of study, year of study, and residency.
As mentioned earlier, the study used the framework from a previous research study in The
Netherlands (van Deursen & van Dijk, 2008). We note the completion rate in our PNG studyof tasks across all four categories, operational, formal, information and strategic is 39, 13, 29
and 28 percent as compared with 80, 72, 62, 22 percent respectively in The Netherlands study.
As reported, the highest percentage of completed tasks was in the lowest category – operational
skills (38%). In this category students were expected to navigate an internet browser, operate
an online search engine and complete an online form. Most of the students were able tocomplete successfully the two initial tasks while the latter task proved the most difficult with
only 3% completing it. Van Deursen and Van Dijk (2008) pointed to a positive correlation
between internet experience and operational skills. This may explain why students in PNG
found it difficult to complete tasks in this category. Comprehensive study of e-government
services in PNG conducted by Daniel (2020) concluded that the majority of the government
websites operate on an emerging level with only basic one-way communication available tothose who use their service; only one-third (about 34%) of the services provided were improved
one-way (e.g. forms for manual completion) and simple two-way communication (e.g. contact
forms and search facilities). Consequently, Daniel observed that the majority of citizens use
services at an emerging level. The demonstrated pattern of internet skills reflects Daniel’s
observation: students were able to navigate websites successfully but only few were able toengage in some form of communication with the service provider.
In the formal skills category, the students were expected to navigate through internet websites
by recognizing and using tabs and hyperlinks without getting disoriented and locating similar
information in different website layouts. Unexpectedly, students did not perform well with only
13% completion across all tasks in this category. The most difficult task involved identifying
physical addresses of four organizations in the capital city. It should be noted that there is no
postal service to physical addresses in PNG. This may help explain why only students from thecapital city, who knew the physical location of the organizations, were able to complete thetask. None of the students from a rural village background completed the task successfully
indicating that life experience could be a significant factor in their ability to complete the digital
task.

119
121

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

The information skills category required students to choose appropriate websites to seek
information. One-third (30%) of students were able to complete all the tasks successfully. The
strategic skills category required students to define search queries and select proper resources.
Almost one-third (28%) of students completed successfully all tasks in this category. We notethat the results in these two categories were almost as high as in operational skills and muchhigher than in formal skills. The results in our study in these two categories show a different
trend from The Netherlands study where higher level of skills showed a lower level ofcompletion. The observed converse trend of successful completion of tasks in the two studies
indicates that long exposure to the internet as in The Netherlands, may well contribute to theoperational and formal skills but is insufficient for successful completion of information andstrategic skills. In PNG, where the exposure to the internet is much shorter, students in theuniversity setting were better able to complete tasks that require intellectual skills. Our
observation remains in line with other studies, which found that the time spent on the internet
has a weak relation with the level of internet skills and influences only operational and formal
skills (van Deursen et al., 2011) while educational level shows a relevant impact on the ability
to solve complex information and strategic tasks on the internet (Godhe, 2019; Gui & Argentin,
2011; Kim, 2019; Passey et al., 2018).
Considering the second objectives, whether there are statistically significant differences among
the categories of gender, age, faculty of study, year of study, and residency, in both places The
Netherlands and PNG there is no significant gender difference. Age did not feature in thesignificance tests for PNG, possibly because all the participants were between 18 and 29 yearsof age.
Considering the year of study, the results demonstrate that the year of study positively
correlates with successful completion of tasks in three categories (Tables 4, 7, and 11). It wasparticularly obvious with only 2% of year one students completing formal skills tasks.
Turning now to the two remaining categories of residency and faculty we note how the PNG
study distinguished between those who came from the capital city, Port Moresby, those whocame from provincial or rural towns, and those from rural villages. This appears to be asignificant predictor, at least for formal and informational skills. It follows that those from
more isolated environments will have less exposure to the digital world and be on the
disadvantaged side of the digital divide.
Similarly, of interest is the significance of Faculty as a predictor in three of four skills for the
PNG students. It may be expected that students in the Faculty of Business Studies andInformatics might have better developed digital skills, but this was apparent only in formal andstrategic skills (Tables 10 and 15). Further study may shed light on why Faculty of Health andMedical Sciences students should score so well in operational skills and so poorly on strategic
skills.
Conclusion
Digital literacy is important in education, particularly tertiary education. Duncan-Howell(2012) has researched the nature of the digital competency exhibited by undergraduate students
in an Australian University. She found that they had a high level of “digital comfort” but that
did not necessarily translate from consuming content to creating content. She concluded that
there is a strong need in higher education “for meaningful use of digital technologies as learning

120
122

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

tools and the development of digital professional skills within programs that is beyond thecurrent practice of being limited to LMS use and email” (p. 838).
At a polytechnic tertiary institution in Singapore, students were found to be not well acquainted
with a range of information search strategies and techniques that would enable them to searcheffectively for information (Laxman, 2009). Laxman proposes school programs that would
enable students to become information literate and more able to fully utilize the educational
computing potential of the internet in developing vibrant, independent learning environments.
The four levels of digital literacy skills from Van Deursen and Van Dijk (2008), provide waysof refining our understanding of digital competency so that we can identify and focus on skills
that need attention. With only 38% of operational tasks and 13% of formal tasks completed in
the PNG study, we cannot conclude as in The Netherlands that such skills will be learned
simply by practice. Effects of the first digital divide mean that many first year students cometo university with limited practice and consequently face a steep learning curve.3 Our research
points to the need for training and formal schooling right from the first year of tertiary
education in PNG, since a good number of students will come with little prior experience.
Divine Word University meets a felt need by giving a laptop to every new fully registeredstudent and providing formal coursework in use of computers and the internet in their firstsemester at the university. The effect of such training may be seen in the relative improvement
between first and second years shown in Table 8. If, as Van Deursen and Van Dijk (2008)
conclude, that “operational and formal skills are a necessary (but not sufficient) condition for
performance of information and strategic skills” (p. 19), then tertiary institutions, particularly
those facing the effects of the digital divide, will need to ensure that those necessary skills areprovided for. Having ensured digital competency at that level, further efforts can be made to
develop information and strategic skills so as to make a meaningful and creative use of digitaltechnologies.

3

Since 2003, Divine Word University offers to all year one students a compulsory unit Introduction to EndUser-Computing and Word that introduces students to the concept of the internet and usage of WWW, issues in
end-user-computing, and basic and intermediate level skills in MS Word.

121
123

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Acilar, A. (2011). Exploring the aspects of digital divide in a developing country. Issues inInforming Science and Information Technology, 8, 231–244. https://doi.org/10.28945/1415
Álvarez-Flores, E. P., Nunez-Gomez, P., & Crespo, C. R. (2017). E-skills acquisition anddeficiencies at the university in the context of the digital economy. Revista Latina de
Comunicacion Social, 72, 540–559. https://doi.org/10.4185/RLCS
Ameen, K., & Gorman, G. E. (2009). Information and digital literacy: A stumbling block todevelopment? A Pakistan perspective. Library Management, 30(1–2), 99–112.
https://doi.org/10.1108/01435120910927565
Ayanso, A., Cho, D. I., & Letrwachara, K. (2011). ICT development index and the digital
divide. International Conference on Information Resources Management.
https://pdfs.semanticscholar.org/2e3e/63fee1a5ab69508954c79d6e077e8e9a19e7.pdf
Chetty, K., Aneja, U., Mishra, V., Gcora, N., & Josie, J. (2018). Bridging the digital divide in
the G20: Skills for the new age. Economics: Open-Assessment E-Journal, 12.
https://doi.org/10.5018/economics-ejournal.ja.2018-24
Chetty, K., Qigui, L., Gcora, N., Josie, J., Wenwei, L., & Fang, C. (2018). Bridging thedigital divide: Measuring digital literacy. Economics: Open-Assessment E-Journal,
12. https://doi.org/http://dx.doi.org/10.5018/economics-ejournal.ja.2018-23
Daniel, M. (2020). Adoption of e-government in Papua New Guinea [unpublished PhD
dissertation]. Divine Word University.
Department of Education. (2008). Computer studies: Upper secondary syllabus. Departmentof Education.
Department of Education. (2016). National education plan 2015-2019: Quality learning forall. Department of Education. https://doi.org/10.1016/j.jqsrt.2014.08.009
Department of Education. (2019). Boosting education standards together in PNG: Education
sector plan implementation grant & multiplier.
https://www.globalpartnership.org/sites/default/files/2019-04-gpe-png-programdocument.pdf
Department of National Planning and Monitoring. (2010). Papua New Guinea medium term
development plan 2011-2015: Building the foundations for prosperity. Department of
National Planning and Development.
Duncan-Howell, J. (2012). Digital mismatch: Expectations and realities of digitalcompetency amongst pre-service education students. Australasian Journal of
Educational Technology, 28(5), 827–840. https://doi.org/10.14742/ajet.819
Eshet-Alkalai, Y. (2004). Digital literacy: A conceptual framework for survival skills in thedigital era. Journal of Educational Multimedia and Hypermedia, 13(1), 93–106.
Eshet-Alkalai, Y. (2012). Thinking in the digital era: A revised model for digital literacy.
Issues in Informing Science and Information Technology, 9, 267–276.
https://doi.org/https://doi.org/10.28945/1621
Fuchs, C., & Horak, E. (2008). Informational capitalism and the digital divide in Africa.
Masaryk University Journal of Law and Technology, 1(2), 11–32.
https://journals.muni.cz/mujlt/article/viewFile/2504/2068
Gašová, K., Mišík, T., & Štofková, Z. (2018). Employers demands on e-skills of university
students in conditions of digital economy. CBU International Conference on
Innovations in Science and Education, 146–152. https://doi.org/10.12955/cbup.v6.1147

122
124

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Gazi, Z. A. (2016). Internalization of digital citizenship for the future of all levels of
education. Education and Science, 186, 137–148.
https://doi.org/10.15390/EB.2016.4533
Godhe, A. L. (2019). Digital literacies or digital competence: Conceptualizations in Nordic
curricula. Media and Communication, 7(2), 25–35.
https://doi.org/10.17645/mac.v7i2.1888
Gui, M., & Argentin, G. (2011). The digital skills of internet-natives: The role of ascriptive
differences in the possession of different forms of digital literacy in a random sample
of northern Italian high school students. New Media & Society, 13(6), 963–980.
https://doi.org/https://doi.org/10.1177/1461444810389751
Halford, S., & Savage, M. (2010). Reconceptualizing digital social inequality. Journal of
Information, Communication & Society, 13(7), 937–955.
https://doi.org/https://doi.org/10.1080/1369118X.2010.499956
Henao, I. (2004). Country report in Papua New Guinea. Pacific Island Forum Secretariat
Sub-Regional Conference on ICT for Development in Pacific Island Developing
Countries, December 6-9.
Ilomäki, L., Paavola, S., Lakkala, M., & Kantosalo, A. (2016). Digital competence: An
emergent boundary concept for policy and educational research. Education andInformation Technologies, 21, 655–679. https://doi.org/10.1007/s10639-014-9346-4
Iordache, C., Mariën, I., & Baelden, D. (2017). Developing digital skills and competences: A
quick-scan analysis of 13 digital literacy models. Italian Journal of Sociology of
Education, 9(1), 6–30. https://doi.org/10.14658/pupj-ijse-2017-1-2
Jiménez-Cortés, R., Vico-Bosch, A., & Rebollo-Catalán, A. (2017). Female university
student’s ICT learning strategies and their influence on digital competence.
International Journal of Educational Technology in Higher Education, 14.
https://doi.org/10.1186/s41239-017-0040-7
Jin, J., & Cheong, A. W. H. (2008). Measuring digital divide: The exploration in Macao.
Observatorio, 6, 259–272. https://doi.org/https://doi.org/10.15847/obsOBS232008186
Katz, I. R. (2007). Testing information literacy in digital environments: ETS’s iskills
assessment. Information Technology and Libraries, September, 1–10.
https://ejournals.bc.edu/index.php/ital/article/view/3271/2884
Kim, K. T. (2019). The structural relationship among digital literacy, learning strategies, andcore competencies among South Korean college students. Educational Sciences:
Theory & Practice, 19(2), 3–21. https://doi.org/10.12738/estp.2019.2.001
Kolodziejczyk, I. (2012). Gender issues in access, application and attitudes toward
information communication technology in higher education institutions in Papua New
Guinea [unpublished PhD dissertation]. Divine Word University.
Komlayut, S., & Srivatanakul, T. (2017). Assessing digital literacy skills using a selfadministered questionnaire. Review of Integrative Business & Economics Research, 6(3), 74–
85. http://buscompress.com/uploads/3/4/9/8/34980536/riber_6-3_a17-026_74-85.pdf
Laxman, K. (2009). A baseline study on the internet information search proficiencies ofpolytechnic students in Singapore. International Journal of Education andDevelopment Using Information and Communication Technology, 5(3).
http://ijedict.dec.uwi.edu/viewarticle.php?id=936&layout=html
Leh, A. S. C., & Kennedy, R. (2004). Instructional and information technology in Papua New
Guinea. Educational Technology Research and Development, 52(1), 96–101.
https://doi.org/https://doi.org/10.1007/BF02504778

123
125

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Lesame, N. (2013). Vision and practice: The South African Information Society experience.
Journal of Multidisciplinary Research, 5(1), 73–90.
Livingstone, S., & Helsper, E. J. (2007). Gradations in digital inclusion: Children, young
people and the digital divide. New Media & Society, 9(4), 671–696.
https://doi.org/10.1177/1461444807080335
Passey, D., Shonfeld, M., Appleby, L., Judge, M., Saito, T., & Smits, A. (2018). Digitalagency: Empowering equity in and through education. Technology, Knowledge andLearning, 23(3), 425–439. https://doi.org/10.1007/s10758-018-9384-x
Ravinder, R. (2011). Challenges for quality primary education in Papua New Guinea: A casestudy. Education Research International, 2011, 1–11.
https://doi.org/10.1155/2011/485634
Reynolds, R. (2016). Defining, designing for, and measuring ‘social constructivist digital
literacy’ development in learners: A proposed framework. Educational Technology
Research and Development, 64, 735–762. https://doi.org/10.1007/s11423-015-9423-4
Salonda, L., Siaguru, P., Yoko, J., & Oliver, W. (2017). A rapid assessment of the Papua
New Guinean graduate labour market to inform the development of Western Pacific
University’s academic programs.
https://www.wpu.ac.pg/images/downloads/Assessment_of_PNG_Labour_Market.pdf
Selvyn, N. (2004). Reconsidering political and popular understandings of the digital divide.
Media & Society, 6(3), 341–362.
https://doi.org/https://doi.org/10.1177/1461444804042519
Spante, M., Hashemi, S. S., Lundin, M., & Algers, A. (2018). Digital competence and digital
literacy in higher education research: Systematic review of concept use. Cogent
Education, 5(1), 1–21. https://doi.org/10.1080/2331186X.2018.1519143
Trucano, M. (2014). Promoting literacy with mobile phones in Papua New Guinea. Blog
Post. https://blogs.worldbank.org/edutech/promoting-literacy-mobile-phones-ruralpapua-new-guinea
Vaa, R. (2003). Papua New Guinea ICT use in education.
http://www.unesco.org/fileadmin/user_upload/ict/Metasurvey/papuanewguinea.pdg
van Deursen, A. J. A. M., Courtois, C., & van Dijk, J. A. G. (2014). Internet skills, sources of
support, and benefiting from internet use. International Journal of Human-Computer
Interaction, 30(4), 278–290. https://doi.org/10.1080/10447318.2013.858458
van Deursen, A. J. A. M., & van Dijk, J. A. G. (2008). Measuring digital skills: Performance
tests of operational, formal, information and strategic internet skills among Dutch
population. ICA Conference, 1–25.
van Deursen, A. J. A. M., van Dijk, J. A. G., & Peters, O. (2011). Rethinking internet skills:
The contribution of gender, age, education, internet experience, and hours online tomedium- and content-related internet skills. Poetics, 39(2), 125–144.
https://doi.org/10.1016/j.poetic.2011.02.001
Corresponding author: Iwona Kolodziejczyk
Contact email: ikolodziejczyk@dwu.ac.pg

124
126

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Teacher Candidate Reflection and Development Through Virtual Exchange
Chesla Ann Lenkaitis
Binghamton University, State University of New York
United States
Shannon M. Hilliker
Binghamton University, State University of New York
United States
Kayla Roumeliotis
Teachers College at Columbia University
United States

125
127

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
This study examines the effects of a virtual exchange on twelve teacher candidates of a
Teaching English to Speakers of Other Languages program. The teacher candidates
participated in a 4-week virtual exchange with English as a Foreign Language learners from a
university in Mexico. Throughout the exchange, the teacher candidates participated inconversational exchanges and subsequently analyzed and reflected on the errors produced by
the English as a Foreign Language students and also the corrective feedback strategies theyused during the sessions. The goal of the analysis and reflection activities was to help theteacher candidates develop their ability to identify such errors and apply appropriate corrective
feedback strategies. The results indicated the teacher candidates’ ability to identify errors
increased throughout the exchange, suggesting changes to their development and perception ofcorrective feedback. These results add to the growing body of research about the value of using
virtual exchanges in teacher preparation programs, a tool that may be particularly relevant
during the current coronavirus global crisis.
Keywords: virtual exchange; learner autonomy; teacher preparation; corrective feedback; adultlearners; language learning

126
128

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Virtual exchange, a practice that partners students from two geographical locations through themeans of technological tools, is an innovative instructional approach being used in higher
education (Bohinski & Leventhal, 2015; Dorner, 2016). The term “virtual exchange” issynonymous with telecollaboration, online intercultural exchanges, and teletandem (O’Dowd,2018). Sadler and Dooly (2016) discussed the increasing use of virtual exchange in language
learning, and how it facilitates communication and promotes learner autonomy. However, theresearch on integrating virtual exchanges in teacher preparation programs is still growing(Lenkaitis, 2020; Dooly & Sadler, 2013; Jauregi & Bañados, 2008).
The current study focuses on teacher candidates in a Teaching English to Speakers of Other
Languages (TESOL) program. In a 4-week virtual exchange, TESOL teacher candidates werepartnered with English as a Foreign Language (EFL) learners. The weekly virtual exchange
sessions gave the EFL learners access to a TESOL teacher candidate with whom they wereable to communicate in English, while providing the teacher candidates opportunities todevelop their pedagogy. The ability to work with an international partner to develop language
and pedagogical skills via virtual exchange is a promising tool that can be valuable as theeducational landscape changes and can address challenges such as the current coronavirus(COVID-19) global crisis. More specifically, this study examines how virtual exchange
sessions created opportunities for TESOL teacher candidates to develop learner autonomy
while learning to recognize and reflect upon EFL student errors and the corresponding
corrective feedback (CF).
Literature Review
Virtual Exchange
Communication is a significant part of learning, especially in the language classroom. Holley
and King (1971) discussed the importance of practicing communication in order to promote a“natural language learning context” (Cohen, 1975, p. 416), where learners are encouraged to
speak and are only corrected for errors that impede communication. Communication andcollaboration can be promoted through the use of technology such as videoconferencing(Fetterman, 1996) and other Web 2.0 tools (Greenhow, Robelia, & Hughes, 2009). Technology
can also facilitate the formation of international partnerships amongst students (Spante et al.,2014) via a virtual exchange. In virtual exchange, a practice that pairs students in other contexts
via technology, students can interact using the target language, and it can be used as an effective
method for improving students’ linguistic development (Ware & O’Dowd, 2008).
Virtual exchange projects have been carried out in English language learning (Austin et al.,2017; Rafieyan et al., 2014; Sevilla-Pavón, 2016) and there is a growing body of research on
how to integrate virtual exchanges in teacher preparation programs (Lenkaitis, 2020; Dooly &Sadler, 2013; Jauregi & Bañados, 2008; The EVALUATE Group, 2019). In Lenkaitis (2020),
participants in a teaching program from a university in the USA were partnered with second
language (L2) learners in a 4-week virtual exchange via videoconferencing. TESOL teacher
candidates explored the language that their L2 partners produced and utilized learned teaching
techniques to interact with course content by watching their recorded sessions. Results revealed
that teacher candidates developed their reflective practices and were able to bridge theory into
practice. In Dooly and Sadler’s (2013) study, student teachers from the USA were partneredwith student teachers from Spain for two years via synchronous (real-time) and asynchronous(non-real-time) means. Results from these collaborations showed that student teachers
recognized the possibilities that exist for L2 learning and teaching through technology, as wellas the benefits of creating materials with other student teachers. In Jauregi and Bañados (2008),

127
129

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

teacher candidates of Spanish from a Chilean university partnered with L2 Spanish learners
from a university in the Netherlands to participate in synchronous video sessions andasynchronous blogs. Analysis of the exchange and questionnaire data indicated that theexchange was beneficial. Not only did teacher candidates and students make connections tocultural topics, but it also allowed them to achieve course outcomes. The EVALUATE Group(2019) was the largest study of teacher candidates that were partnered in virtual exchanges todate. Over 1,000 teacher candidates from over thirty countries formed twenty-five partnerships
with the goal of completing tasks that included developing curriculum and educational
materials. Qualitative and quantitative results revealed that competences, such as digitalpedagogical, language, and intercultural, developed as a result of the collaboration.
In their review of virtual exchange studies, Akiyama and Cunningham (2018) expressed that
only 13% of all studies have focused on partnerships between foreign language (FL) learners
and native speaker (NS) teachers in training. Furthermore, only 20% of studies have centredaround FL learners and non-native speakers (NNS). In this type of exchange known as anapprenticeship exchange, one group utilizes teaching strategies while the other group learns aforeign language. Therefore, the exchange is typically in the target language of the FL learners(Chaudhuri, 2011; Jauregi & Bañados, 2008). While Akiyama and Cunningham (2018)
mentioned apprenticeship exchange as a typical configuration for a virtual exchange, thisarticle focuses on a study whose participants consisted of teacher candidates that were both NS
and NNSs of English.
Learner Autonomy
Because learner autonomy is encouraged by virtual exchange (Sadler & Dooly, 2016) and is acentral concept of language teaching and learning (Benson, 2013; Holec, 1981; Little, 1998),
this study focused on the learner autonomy of TESOL teacher candidates. Autonomous
learners, such as the TESOL teacher candidates in this study, are fully responsible for thelearning process including reflection and analysis to plan, monitor and evaluate learning (Little,
1998). Little (1998) discussed the importance of learner autonomy in language learning.
According to Little (1998), there are three basic pedagogical principles to learner autonomy:
learners must be involved in the learning process; learners must reflect and evaluate theirlearning; and learners must use the appropriate target language.
Self-assessment and reflection play a significant role in learner autonomy. Through reflection,
learners can recognize the methods and strategies they use and, as a result, evaluate theirlearning, identify any problems and suggest solutions (Çakici, 2015). A teacher’s role in alearner autonomous language classroom is different from a traditional teacher’s role, in thatteachers can be described as a facilitator of learning (Little, 1995). Teachers and students are
partners working towards a common goal. Teachers teach their students how to learn byoffering different methods of learning and involving students in decision making processes(Çakici, 2015). Little (1995) stated that successful teachers are autonomous in that they practicecontinuous reflection and analysis of the teaching process. This practice is useful in thelanguage classroom when teachers keep a record of learners’ errors and the feedback given to
students. This log of information can help teachers understand what errors are commonly madeby students and what feedback has the most beneficial effect. When reflecting on corrective
feedback (CF) strategies, teachers can gain a greater understanding of the effectiveness of astrategy and even change their practices as a result (Ellis, 2009).
Although learner autonomy encompasses activities done individually, another component ofautonomous learning is group work. Çakici (2015) discussed the importance of cooperative

128
130

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

learning in an autonomous classroom where “its aim is to establish a community of learners inwhich students are able to generate questions and discuss ideas freely with the teacher and eachother” (p. 36). Cooperative learning has also been proven to result in higher self-esteem,
confidence and rapid achievement (Çakici, 2015). Virtual exchange may foster this cooperative
learning because student pairs are working together towards a common goal. Collaboration
between students, teachers and schools has always been an essential part of education, and now
with the use of technology and the increase of internet connections, it is easier to connect andwork with others (Dooly, 2017) and learn beyond the classroom (Reinders & Benson, 2017).
Corrective Feedback (CF)
Reflecting on CF strategies can be an autonomous activity completed by teachers. There aremany factors that can affect whether a CF strategy is effective, so teacher education programs
are normally reluctant to tell teacher candidates what strategy to use (Ellis, 2009). Therefore,
a common issue in language teaching is the role of error correction (Chenoweth, 1983).
Language teachers can facilitate learning by having a greater understanding of the importanceof learners’ errors and when it is necessary to make a correction. Furthermore, making errors
is a significant part of learning and it can be most beneficial to students when errors arefollowed by CF (Metcalfe, 2017). However, it is crucial that a teacher’s method of correction
encourages an environment in which students are not afraid to make mistakes, one that
promotes “students’ active, exploratory, generative engagement” (Metcalf, 2017, p. 61).
Gregersen (2003) stated that although making errors is a necessary part of the learning process,it is also important to realize how learners will react to errors and corrections. Students tend toprefer receiving CF for their errors rather than ignoring them, however, learners’ cultural
backgrounds, language learning experiences, and proficiency levels play a role in theirpreferences as well (Lyster et al., 2013). CF is not only a pedagogical tool for teachers, but also
a motivational tool for language learners. Language learners can improve their linguistic
performance through supportive CF (Ito & Hilliker, 2018). Teachers often express a preference
for only correcting errors that impede communication as to not disrupt the flow ofcommunication or discourage students (Jean & Simard, 2011; Lyster et al., 2013).
Ellis (2009) discussed different CF strategies with definitions and examples. Types of CF vary
based on the degree of implicitness or explicitness. Explicit CF is when the corrector identifies
the error and may also provide a correction. In contrast, implicit CF allows for the student toself-correct. One controversy in error correction is the degree to which a correction should be
explicit. Lyster et al. (2013) discussed research suggesting that implicit CF may have a longer
lasting effect than explicit CF (Mackey & Goo, 2007; Li, 2010), although explicit CF is more
noticeable to learners (Mackey et al., 2007; Nassaji, 2009).
Implicit CF strategies include recasts, repetition, and clarification requests. These methods are
similar to when children learn their first language and parents hint at the correct response. Ellis(2009) provided clear definitions for the different CF strategies. Recasting is when the corrector
reformulates the student’s utterance preceding the error to change or correct in some way.
According to Lyster et al. (2013), recasts are common and “well suited in communicativeclassroom discourse because they tend to not interrupt the flow of communication” (p. 10).
Repetition is when the corrector repeats the student’s utterance putting an emphasis on theerror. A clarification request is when the corrector indicates a misunderstanding. On the other
hand, elicitation and paralinguistic signals are examples of explicit CF strategies. Withelicitation, the corrector indicates that an error is made by using intonation in the repetition of

129
131

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

the student’s utterance. Paralinguistic signals are gestures or facial expressions that indicate anerror.
Research Questions
Because reflecting on CF is critical in developing TESOL teacher candidates’ pedagogy andvirtual exchange can be a vehicle for this autonomous activity, the current study aims to
examine how a teacher preparation program’s use of virtual exchange promotes learnerautonomy and teacher candidates’ CF development. Therefore, a study was conducted toanswer the following research questions:
1. Upon reflecting on their participation in a virtual exchange, in what ways do TESOL
teacher candidates recognize EFL learners’ errors during the exchange?
2. Upon reflecting on their participation in a virtual exchange, in what ways do TESOL
teacher candidates develop CF strategies during the exchange?
Methods
Participants
In total, twelve TESOL teacher candidates participated in the study. All of these participants
were registered for a Linguistics for Teachers course at a university in the USA. The course
topics included pragmatics, syntax, semantics, morphology and phonology. As part of required
coursework, TESOL teacher candidate participants were partnered with 1-2 EFL learners froma Mexican university. The average age of the TESOL teacher candidates was 29.5 years old(SD = 8.76).
Procedures
Echoing Lenkaitis (2020), participants were given instructions to video conference with their
EFL partner(s) for at least 20 minutes for four weeks (Week 1-4). As a way to promoteauthentic practice, the TESOL teacher candidates implemented strategies they were learningin their Linguistics for Teachers class. These strategies aimed to help develop their partners’
English skills during the synchronous sessions, which were done through Zoom
(https://zoom.us). There were no weekly topics given to the partnerships, as the weekly
synchronous sessions were meant to be authentic conversations between the EFL learners andthe TESOL teacher candidates.
After each video conferencing session, the TESOL teacher candidates were asked to watch
their recorded synchronous session. While watching these sessions, the teacher candidates
identified and recorded the EFLs errors on a weekly chart, along with the time of the occurrence
and the CF strategies they implemented during the session. If the TESOL teacher candidates
did not implement a CF strategy, they were asked to write an idea of a CF strategy that could
have been used at that time. For the two weeks following the video conferencing sessions, theteacher candidates were asked to revisit one of the four sessions, which involved rewatching
the video recording and identifying and reflecting on the EFL learners’ errors (Week 5 and 6).
In addition, the TESOL teacher candidates were required to write a final reflection on theirvirtual exchange experience and video reflections (Week 7). Table 1 details all seven weeks ofthe current study.

130
132

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Table 1: Teacher Candidate Activities and Output by Week
Week(s)
1–4
5–6
7

Activity
Video conference with EFL learners for atleast 20 minutes each week
Revisit one of the four weekly
synchronous sessions to review ELF
learners’ errors
Complete final reflection

OutputVideo recordings
Chart of ELF errors
List of CF strategies
Written reflection

Data Collection and Analysis
Researchers coded the teacher candidates’ reflections in order to see whether their recognitionof errors changed or stayed the same throughout the exchange. In order to see when the teacher
candidates found it easier to recognize errors, researchers coded reflections into three
categories: Difficult, Easy, and Unknown. The Difficult and Easy categories had to do withwhether the TESOL teacher candidates’ expressed difficulty or ease in recognizing errors. The
unknown category was for when participants expressed confusion in recognizing EFL learners’
errors. In order to illustrate how teacher candidates’ perceptions about CF developed
throughout the virtual exchange, researchers coded the TESOL teacher candidates’ charts andjournal reflections into specific categories using NVivo 11. These categories included Need for
Implicit Feedback, Need for Explicit Feedback, No Need for Implicit Feedback, No Need for
Implicit Feedback, and Unknown (see Table 2 for a summary). As mentioned in the literature
review, both Need and No Need categories were important for data collection and analysis dueto the ongoing debate about the role of error correction in language learning. Coding wascompleted by two researchers with an 89.6% interrater reliability (Kappa = 0.67 with p <0.001). To reconcile differences, both worked together to reach a 100% agreement rate.
Table 2: Name and Description of the Corrective Feedback Categories Coded by Researchers
Need for Implicit
Feedback
Need for Explicit
Feedback
No Need for Implicit
Feedback
No Need for Explicit
FeedbackUnknown

Teacher candidate talks about needing to …
● encourage students to speak freely● give implicit CF (recasts, repetitions, clarification requests
and elicitation)
Teacher candidate talks about needing to give EFL learners● explicit CF● lesson ideas for correction
Teacher candidate does not talk about needing to give EFL learners● encourage students to speak freely● implicit CF (recasts, repetitions, clarification requests andelicitation)
Teacher candidate does not talk about needing to give EFL learners● explicit CF● lesson ideas for correction
Teacher candidate talks about not knowing what kind of CF the ELF
learners need
Results

All of the study’s participants completed the weekly assignments and final reflection. Eachvideo session averaged 32 minutes and 30 seconds. Figure 1 details the average number of

131
133

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

errors found by the TESOL teacher candidates in the EFL learners’ speech during each weekly
synchronous session. As shown in Figure 1, the teacher candidates’ recognition of errors
increased in the first four weeks. In addition, their recognition of errors increased again inweeks 5 and 6, when they reanalyzed the recorded video sessions. For example, from Week 1
to Week 6, the average number of errors increased from 10.56 (SD = 8.92) to 19.25 (SD =19.8).
25.00
20.00
15.00
10.00
5.00
0.00

Week Week Week Week Week Week
1
2
3
4
5
6

Average Number
10.56 11.00 16.25 15.25 16.08 19.25
of Errors
Standard Deviation 8.92 10.98 17.64 16.14 18.29 19.80
Figure 1: Average EFL Student Errors
In addition to reporting these errors, researchers identified the most frequent words used in the
TESOL teacher candidates’ weekly and final journals, as well as their weekly and final charts.
Many of the common words found in the journal reflections were related to use of language(e.g., word, conversation, understand, sentence). The common words found in the charts reflect
linguistic categories, errors and corrections (e.g., syntax, semantics, morphology, error,
correct). The bolded words in Table 3 highlight the most common words found across all
documents (charts and journals).
Table 3: Ten Most Frequent Words Used by Teacher Candidates by Week Produced
Charts and Journals
Produced in Weeks 1–4
Like
Correct
Word
Sentence
Using
Question
Students
Semantics
Times
SyntaxChart and Journals
Produced in Weeks 5–6
SyntaxErrorsStudents
Zoom
Learning
SessionLanguageMorphology
English
Use

132
134

Charts and Journals
Produced in Weeks 1–7
Like
Correct
Sentence
Word
Using
Students
Question
SyntaxTimes
Semantics

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

The teacher candidates expressed ease and difficulty recognizing EFL learners’ errors and how
it varied throughout the exchange. Researchers found that the total number of coding instances
for difficult (11) and easy (10) were around the same as seen in Table 4. However, based on
the number of coding instances for each week, one can see that participants spoke about it
being more difficult in the beginning than at the end of the exchange.
Table 4: Number of Coded Instances by Week
Codingcategory
Difficult
Easy
Unknown

1

2

Week of Study
3
4
5

1
1
1

2
0
0

2
0
3

1
0
2

0
3
0

6

7

Total

0
3
0

5
5
3

11
12
9

The number of instances for the implicit/explicit categories across the weeks of the exchange
are shown in Table 5. Teacher candidates expressed a greater need for implicit feedback than
explicit feedback as there were 144 coded instances for explicit feedback versus 28 for implicit
feedback. In addition, the no need for explicit feedback category had a greater number of coded
instances (53) than the no need for implicit feedback category (37). This further demonstrates
the need for implicit feedback.
Table 5: Number of Coded Instances for Need for Implicit and Explicit Feedback
Need for
Explicit

Need for
Implicit

No need forExplicit

No need for
Implicit

Unknown

Week
1

Chart

1

8

2

2

0

Journal

0

12

11

4

0

Week
2

Chart

0

14

4

2

2

Journal

2

6

4

3

0

Week
3

Chart

3

17

8

7

1

Journal

1

2

1

1

1

Week
4

Chart

4

24

6

5

1

Journal

2

1

3

3

0

Week
5

Chart

7

21

4

4

0

Journal

2

4

2

1

0

Week
6

Chart

1

18

6

4

0

Journal

5

5

0

0

1

Chart

0

10

0

0

0

133
135

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Week
7

Journal

0

2

2

1

3

TotalsAll Charts

16

112

30

24

4

All Journals

12

32

23

13

5

All
documents

28

144

53

37

9

Discussion
Recognition of EFL Learners’ Errors
During the virtual exchange, recording the video conference sessions allowed teachers torevisit their weekly synchronous meetings in order to reflect on their pedagogy. Just as Kessler
and Hubbard (2017) discussed the benefits of technology for observations and monitoring ofstudent behaviour and progress, noting that teachers can also gain insight on the identificationof “linguistic and technological challenges students face, the current study’s practices can also
contribute to the design of more salient feedback” (Kessler & Hubbard, 2017, p. 284). In thisvirtual exchange study, through practice and acknowledgement of the importance of errors,
TESOL teacher candidates began to recognize more errors in their EFL learners’ speech.
The number of errors identified weekly as recorded in the charts increased throughout thevirtual exchange, and many participants also stated in their reflections that the recognition oferrors became easier each week. For instance, participant 2 in Week 6 wrote that, “finding theerrors or examples of topics we discussed in class seemed to be more evident than when I
analyzed it the first time through”. Teacher candidates were both NS and NNS of English andso their difficulties differed in that the NNSs expressed more difficulty in recognizing EFL
learners’ errors. This could be because English is also not their first language and it wasdifficult for them to understand what their partner was saying.
The number of coded instances for “difficult” and “easy” had similar totals. However, looking
at the distribution of the coded instances throughout the exchange demonstrates that the teacher
candidates spoke about their difficulty in recognizing and identifying errors in the beginningof the exchange just as much as they spoke about how they found it easier by the end of theexchange. In his/her final reflection, participant 1 noted, “It was really overwhelming at firstto find and classify errors since I didn’t understand my conversation partner really well.
However, the process became easier over time, and I was able to come up with teaching
recommendations more easily”. As a result of keeping a record of their students’ errors, thevirtual exchange allowed teacher candidates to identify recurring errors that their partners madeand to recognize the importance of correcting those recurring errors.
Development of Corrective Feedback Strategies
Reflection, being a significant part of autonomous learning (Little, 1995), was implemented
into this virtual exchange as teacher candidates were asked to reflect on their weekly
videoconferences. Researchers found that participants wrote about the nature of the exchange,
EFL learners’ progress, the errors they found in their EFL learners’ speech, and the use of CF.
Throughout this virtual exchange, participants reflected on their weekly meetings anddeveloped methods of CF in an attempt to facilitate communication with their partner and allow
for improvement in the EFL learners’ speech.

134
136

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

In the first week, teacher candidates were more reluctant to give explicit CF. For example,
participant 11 said, “I generally feel like I wouldn’t overtly correct a student’s errors unless
those errors impeded understanding”” This is similar to Jean and Simard’s (2011) finding thatteachers preferred correcting errors that impede communication. Some possible reasons forreluctance around providing explicit feedback might include that it was their first time meeting
their partner and they wanted to encourage their partners’ willingness to converse. Anotherexplanation is the teacher candidates were not able to easily identify their partners’ errors.
Teacher candidates also wrote about the CF strategies they used in the charts they created.
Candidates who did not use any CF strategies during the video exchanges wrote their CF ideas
for they might have done. As the weeks progressed, teacher candidates developed more implicit
CF strategies and demonstrated a greater need for implicit CF than explicit. Examples of
implicit CF include, recasts, repetitions, clarification requests and elicitation. Participant 5
discussed the strategies he/she found to be useful in the exchange, “I adopted several methods
for elucidating meaning: reformulating my question/response, adding additional information,using simplified vocabulary and statements, or incorporating known vocabulary”.
Perceptions of Corrective Error Feedback
Reflections showed that teacher candidates’ views on CF strategies shifted across their weekly
exchanges. Similar to Vasquez and Harvey (2010), teacher candidates in this study reflectedon their recorded lessons and, as a result, developed a greater understanding of the role andfunction of CF and the interaction between CF and learner uptake. Overall, most teacher
candidates expressed a greater need for implicit CF than explicit. However, some participants’views changed towards the end of the exchange when they expressed a need for explicit CF.
In Weeks 5 and 6 when they revisited and reflected on a session of their choice, it was evident
that the teacher candidates began to recognize more errors in their EFL learners’ speech.
Participants who expressed a need for explicit CF during these weeks, spoke about how it could
be a possible solution for correcting recurring errors in their EFL learners’ speech. Participant
11 wrote, “the more I pay attention to my conversations with [my partner], the more I noticethat his most frequent errors relate to plurals and non-plurals, pronunciation, and tense. Given
this information, if I were to continue meeting with [him], I might plan an informal lesson orconversation topic surrounding these areas, so that he could improve”.
Participants who did not express a need for explicit CF continued to develop a preference for
implicit CF. For example, participant 2 stated, “After learning how to analyze speech and detect
errors, I was able to see where my conversation partner needed extra support, though we didn’t
reach the stage where I felt it was appropriate to correct her errors”. These reflections also
pointed out that the length of the virtual exchange could have played a role in the use of implicit
rather than explicit CF. In a longer exchange, teachers may have felt more comfortable withmaking explicit corrections and more likely to identify recurring errors.
Limitations
Having a virtual exchange consisting of more than four weeks of synchronous sessions would
be helpful for considering the ways in which CF strategies change over time and to examine
the role the length the exchange plays in the pedagogical development of teacher candidates.
In addition, because the data were self-reported, some participants may have not wanted to
fully express their ideas, which in turn could have impacted their reflections. Regardless,coding was completed in order to find commonalities and the two independent coders worked

135
137

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

together to choose the most representative examples from the analysis. Finally, examining
similarities and differences between teacher candidates of other content areas could be
beneficial to examine the value of virtual exchanges with these other areas.
Conclusion
In this study, a virtual exchange gave TESOL teacher candidates authentic practice with EFL
learners and, in turn, a greater sense of responsibility for their learning (Sadler & Dooly, 2016).
In this study, teacher candidates were offered different methods of learning in which they wereable to participate in decision making processes and take control of their learning (Çakici,
2015). As a result of this learning and the autonomous activities, including self-assessment andreflection, the teacher candidates formed a greater recognition of EFL learners’ errors anddevelopment of their CF strategies. Overall, this study found that teacher candidates’
recognition of errors improved, and their reflections demonstrated how this became easierthroughout the exchange. Teacher candidates also demonstrated a greater need for implicit CF
for their students and how their perceptions of CF strategies changed depending on theirstudent’s recurring errors. Based on the results of this study, which partnered TESOL teacher
candidates in a virtual exchange with EFL learners, it seems clear that the following
recommendation should be made: Virtual exchange opportunities and subsequent reflection
should be integrated into teacher preparation programs. Not only will the utilization of virtual
exchange give teacher candidates the opportunity to work with learners in their content area,
but also to develop their CF strategies. Furthermore, having the ability to work with aninternational partner to develop these skills showed that virtual exchange is a tool that can beeven more valuable in the era of digital transformation that has been highlighted during the
COVID-19 crisis.
Acknowledgments
In carrying out this study, researchers received assistance from the following colleague, whodeserves our thanks: Salvador Venegas Escobar.

136
138

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Akiyama, Y. & Cunningham, D.J. (2018). Synthesizing the practice of SCMC-based
telecollaboration: A scoping review. CALICO Journal, 35(1), 49–76.
https://doi.org/:10.1558/cj.33156
Austin, N., Hampel, R. & Kukulska-Hulme, A. (2017). Video conferencing and multimodal
expression of voice: Children's conversations using Skype for second language
development in a telecollaborative setting. SYSTEM, 64, 87–103.
https://doi.org/:10.1016/j.system.2016.12.003
Benson, P. (2013). Learner autonomy. TESOL Quarterly, 47(4), 839–843.
https://doi.org/ 10.1002/tesq.134
Bohinski, C. A. & Leventhal, Y. (2015). Rethinking the ICC framework: Transformation andtelecollaboration. Foreign Language Annals, 48(3), 521–534.
https://doi.org/10.1111/flan.12149
Çakici, D. (2015). Autonomy in language teaching and learning process. İnönü ÜniversitesiEğitim Fakültesi Dergisi [Inonu University Journal of the Faculty of
Education], 16(1), 31–42. https://doi.org/doi:10.17679/iuefd.16168538
Chaudhuri, T. (2011). Designing Web 2.0-telecollaborations for university students. The
eExchange Giessen–Hong Kong. German as a Foreign Language, 12(2), 126–141.
Retrieved from http://www.gfl-journal.de/2-2011/chaudhuri.pdf
Cohen, A. D. (1975). Error correction and the training of language teachers. The Modern
Language Journal, 59(8), 414–422.
https://doi.org/10.1111/j.1540-4781.1975.tb04722.x
Dooly, M. (2017). Telecollaboration. In C.A. Chapelle & S. Sauro (Eds.), The handbook oftechnology and second language teaching and learning. (pp. 169–183). Wiley.
https://doi.org/10.1002/9781118914069.ch12
Dooly, M. & Sadler, R. (2013). Filling in the gaps: Linking theory and practice throughtelecollaboration in teacher education. ReCALL, 25(1), 4–29.
https://doi.org/10.1017/S0958344012000237
Dorner, H. (2018) Exploring students’ conceptions of internationalised learning: Experiencesfrom international online collaborative seminars, Innovations in Education andTeaching International, 55(3), 304–313.
https://doi.org/10.1080/14703297.2016.1210530
Ellis, R. (2009). Corrective feedback and teacher development. L2 Journal, 1(1), 3–18.
https://doi.org/10.5070/l2.v1i1.9054
Fetterman, D. M. (1996). Videoconferencing on-line: Enhancing communication over theInternet. Educational Researcher, 25(4), 23–27.
https://doi.org/10.3102/0013189X025004023
Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in adigital age: Web 2.0 and classroom research: What path should we take now?.
Educational Researcher, 38(4), 246–259. https://doi.org/10.3102/0013189X09336671
Gregersen, T. S. (2003). To err is human: A reminder to teachers of language‐anxious
students. Foreign Language Annals, 36(1), 25–32.
https://doi.org/10.1111/j.1944-9720.2003.tb01929.x

137
139

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Holec, H. (1981) Autonomy and foreign language learning. Pergamon.
Holley, F. M., & King, J. K. (1971). Imitation and Correction in Foreign Language
Learning. The Modern Language Journal, 55(8), 494–498.
https://doi.org/10.1111/j.1540-4781.1971.tb04612.x
Ito, K. & Hilliker, S. M. (2018). Supporting university students through supportive ESL
instruction and corrective feedback. In B. Blummer, J. Kenton and M. Wiatrowski(Eds.), Promoting ethnic diversity and multiculturalism in higher education (pp. 40–
58). IGI Global.
Jauregi, K. & Bañados, E. (2008). Virtual interaction through video-web communication: A
step towards enriching and internationalizing language learning programs. ReCALL,
20(2), 183–207. https://doi.org/10.1017/S0958344008000529
Jean, G. & D. Simard (2011). Grammar learning in English and French L2: Students’ andteachers’ beliefs and perceptions. Foreign Language Annals 44(4), 465–492.
https://doi.org/10.1111/j.1944-9720.2011.01143.x
Kessler, G. & Hubbard, P. (2017). Language teacher education and technology. In C. A.
Chapelle and S. Sauro (Eds.), The handbook of technology and second language
teaching and learning (pp. 278–292). Wiley.
https://doi.org:10.1002/9781118914069.ch19
Lenkaitis, C. A. (2020). Teacher candidate reflection: Benefits of using a synchronous
computer-mediated communication-based virtual exchange. Teaching and Teacher
Education. https://doi.org/10.1016/j.tate.2020.103041
Li, S. (2010). The effectiveness of corrective feedback in SLA: A meta-analysis. Language
Learning, 60(2), 309–365. https://doi.org/10.1111/j.1467-9922.2010.00561.x
Lyster, R., Saito, K. & Sato, M. (2013). Oral corrective feedback in second language
classrooms. Language Teaching, 46(1), 1–40.
https://doi.org/10.1017/S0261444812000365
Mackey, A., M. Al-Khalil, G. Atanassova, M. Hama, A. Logan-Terry & K. Nakatsukasa(2007). Teachers’ intentions and learners’ perceptions about corrective feedback in
the L2 classroom. Innovation in Language Learning and Teaching, 1(1), 129–152.
https://doi.org/10.2167/illt047.0
Mackey, A. & J. Goo (2007). Interaction research in SLA: A meta-analysis and research
synthesis. In A. Mackey (Ed.), Conversational interaction in second language
acquisition: A series of empirical studies (pp. 407–453). Oxford University Press.
Metcalfe, J. (2017). Learning from errors. Annual Review of Psychology, 68, 465–489.
https://doi.org/10.1146/annurev-psych-010416-044022
O’Dowd, R. (2018). From telecollaboration to virtual exchange: state-of-the-art and the roleof UNICollaboration in moving forward. Journal of Virtual Exchange, 1, 1–23.
https://doi.org:10.14705/rpnet.2018.jve.1
Rafieyan, V., Sharafi-Nejad, M., Khavari, Z., Siew Eng, L., & Rashid Mohamed, A. (2014).
Pragmatic comprehension development through telecollaboration. English Language
Teaching, 7(2), 11–19. https://doi.org/10.5539/elt.v7n2p11
Reinders, H., & Benson, P. (2017). Research agenda: Language learning beyond theclassroom. Language Teaching, 50(4), 561–578.
https://doi.org/10.1017/S0261444817000192

138
140

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Sadler, R. & Dooly, M. (2016). Twelve years of telecollaboration: what we have learnt. ELT
Journal, 70(4), 401–413. https://doi.org/10.1093/elt/ccw041
Sevilla-Pavón, A. (2016). Affordances of telecollaboration tools for English for SpecificPurposes online learning. World Journal on Educational Technology, 8(3), 218–223.
https://doi.org/10.18844/wjet.v8i3.696
Spante, M., Karlsen, A. V., Nortvig, A.-M., & Christiansen, R. B. (2014). Cross-Bordercollaboration in history among Nordic students: A case study about creating
innovative ICT didactic models. IAFOR Journal of Education, 2(2).
https://doi.org/10.22492/ije.2.2.02
The EVALUATE Group. (2019). Evaluating the impact of virtual exchange on initial teacher
education: A European policy experiment. Research-publishing.net.
https://doi.org/10.14705/rpnet.2019.29.9782490057337
Vásquez, C. & Harvey, J. (2010). Raising teachers’ awareness about corrective feedback
through research replication. Language Teaching Research, 14(4), 421–443.
https://doi.org/10.1177/1362168810375365
Ware, P. & O'Dowd, R. (2008). Peer feedback on language form in telecollaboration.
Language Learning & Technology, 12(1), 43–63. https://doi.org/10125/44130
Corresponding author: Chesla Ann Lenkaitis
Contact email: chesla.ann.lenkaitis@gmail.com

139
141

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

140
142

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Training Factors as Predictors of Students’ Self-Efficacy Beliefs
for Online Journalism PracticeJoseph Njuguna
University of Rwanda
Rwanda141
143

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
The advent of Internet technologies has heralded new skill demands in the media industry.
Since journalism education mainly takes its cue from industry trends, media training
institutions are now forced to adjust their curricula and teaching styles to produce online-readygraduates. Drawing on aspects of self-efficacy theory, this correlation study employs aquestionnaire to explore how different training factors influence students’ self-efficacy beliefs
for online journalism work. A sample of 182 mass communication students from five Rwandan
universities participated in the study. Results showed that the training factors explained 29.7%of the variance in the respondents’ self-efficacy beliefs for online journalism work, withpositive correlations between all the training factors and the students’ self-efficacy. In
particular, the types of online skills (β =.069) and availability of teaching facilities (β=.076)
contributed a larger part of the online self-efficacy beliefs than teaching styles (β=.018). These
results showed that training factors have a role in boosting students’ beliefs in their capacity to
execute online journalism tasks in the industry. Results suggested that journalism educators
especially need to enhance different online journalism teaching approaches in order to better
develop future professionals who are online-confident for the workplace.
Keywords: online journalism, self-efficacy, training, pedagogy, Rwanda

142
144

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

One defining feature of journalism education is the ability of graduates to demonstratecompetence in the changing work environment (Schwalbe, 2009). With Web 2.0 technologies,
professional news production has been transformed as news audiences (called “citizen
journalists”) become active producers and consumers of news, inevitably putting the relevanceof journalism education into sharper focus (Oluchi, 2016). Furthermore, the industry’s gradualinclination towards multi-skilled and techno-savvy graduates has presented a new reality formedia training institutions who are forced to adapt by re-designing their curricula and adopting
better ways of teaching these new skills (Robinson, 2013; Quinn, 2010). Short of this,journalism education risks playing the technological catch-up game as stakeholders continue
to wonder if the institutions are adequately prepared to develop online-ready graduates(Ehreneich, 2009).
Educating the new generation of media professionals requires breaking from traditional
teacher-led approaches by taking cognizance of students’ exposure and agility with the digital
tools they are expected to use in the industry (Patrão & Figueiredo, 2018; Iordache et al., 2017).
With traditional media models almost obsolete, it has been argued that journalism education
can only extricate itself from the crisis of relevance by devising new ways of integrating
technology in the students’ learning experiences. Particularly, the traditional “silo” teaching ofTV, radio and print separately has been challenged, with scholars advocating for an integrativeuse of the technologies to teach these skills for a multimedia environment (Kaul, 2013; Quinn,
2010; Iyer (2015). Harnessing social media tools to co-produce and share story pieces throughclass blogs and Facebook pages are becoming the teaching norm in a number of institutions(Patrão & Figueiredo, 2018). In contrast with traditional journalism pedagogy, co-creation ofstories affords students valuable learning moments, especially through accompanying online
comments to their productions. This teaching approach also scaffolds learners with an active
community of practice enabling them to authenticate their learning experiences (Cindy, 2015;
Ferrucci, 2017).
In online skills training, the nature of content, training approaches and facilities have been
considered crucial in shaping the acquisition of skills for the industry (Iordache et al., 2017;
Iyer, 2015; Jeanti, 2015). In the journalism context, however, research is needed to explore
how students perceive their ability to use these skills at the work place. Some studies have
explored students’ journalistic writing self-efficacy (e.g. Broaddus, 2012) and efficacy beliefs
in social media use (e.g. Patrão & Figueiredo, 2018). Focus on the role of training factors on
students’ online journalism self-efficacy has not been systematically examined. Although
Rwandan journalism schools have integrated the teaching of online journalism skills in theircurricula, it remains unclear how the training contributes to students’ confidence in the use ofsuch skills when they graduate. Hence, this study hopes to fill this void by exploring the roleof content, training approaches and facilities in the students’ self-efficacy beliefs for online
journalism work.
Literature Review
Digital skills are considered critical for professional survival in an increasingly e-permeatedsociety. Indeed, Ferrari (2012) considers digital competence as one of the critical competences
for life-long learning, comprising the “knowledge, skills, attitudes, abilities, …required whenusing digital media to perform tasks …” (p.11). In light of the fast-evolving technologies,
journalism scholars argue that clarity of the variety of online skills required for digital workreadiness among the future professionals is urgently needed (Iyer, 2015; Jeanti, 2015).
However, in the absence of a coherent framework for such skills, Gallardo-Echenique et al.

143
145

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020(2015) suggest that a digital competence framework like Ferrari’s (2012), which amalgamatesother frameworks, can be contextualized to help define the skills expected from media
professionals. This framework expects the 21st century professionals to have skills in social
media communication, creation and distribution of multimedia content, online ethics, online
research and using social media to solve communication problems.
Evidence suggests that skills like online research, multimedia content creation and social media
communication are the cornerstone of modern journalism practice (Jeanti, 2015; Cindy, 2015).
The ability to produce stories for different formats and share them on social media networks
such as blogs, Twitter or news websites has become crucial. Other critical skills gaining
currency include the ability to work with web code and data analytics, as well as crowd
sourcing stories through artificial intelligence-enabled techniques. In essence, the burgeoning
skills call for innovative ways of using the technology to maximize news audience satisfaction(Robinson, 2013; Hirst & Treadwell, 2011).
Journalism educators are urged to tune their training content and techniques by adopting alearning environment that replicates the digital newsroom, which is expected to enhance
students’ digital production skills (Iyer, 2015; Hirst & Treadwell, 2011). Evidence shows that
if social media tools are well appropriated in journalistic training, online story-telling practicesof the future professionals can be enhanced (Aifan, 2015; Hirst & Treadwell, 2011; Quinn(2010). Although a connected multimedia environment with unlimited opportunities forpractice is the ideal for online production skills, leveraging the near-ubiquitous mobile phones
and free online software for editing has sufficed to circumvent the costly equipment in a
number of schools (Salaverría, 2011; Kaul, 2013). Indeed, Bethell (2010) argues thatjournalism students only need a mobile phone and curiosity to produce professional-readystories. Researchers argue that if students have the necessary environment to develop a skill,they will gradually experience confidence and success with the skill and tasks associated withthat skill. Bandura (1986) termed this feeling or belief in their ability to perform a task as one’s
self-efficacy.
While self-efficacy in the context of journalistic performance has largely focused on writing
skills, little attention has been given to new media skills (Broaddus, 2012). Becker et al. (2012)
surveyed the journalism work efficacy of 2,195 US graduating students. More than 70% ofthem attributed the content taught for their self-efficacy for writing and editing for the web,creating blogs, and use of the social media professionally. Wotkyns (2014) explored Australian
university students’ satisfaction of the training environment in a new convergent journalism
major. 88% were extremely-moderately satisfied with the online skills given, indicating thatstudents’ learning expectations were being met. A study by Huang et al. (2020) found thatstudents’ journalistic writing self-efficacy was positively correlated with their actual writing
performance, although the relationship was weak (r =.16), suggesting that their unfamiliarity
with the unique requirements of news writing could have been the cause.
Broaddus (2012) explored how learning strategies (among other factors) contributed to US
journalism majors’ writing self-efficacy. Practical classroom assignments contributed more in
students’ writing self-efficacy than the students’ background experiences and their general
writing background. A linear regression analysis on the extent to which learning experienceswith such scholastic media associated with writing self-efficacy indicated a statisticallysignificant model (F4, 445 = 10.075; p = .000; R2= .075) where newspaper experience (t =3.678; p = .000) and journalism classes (t = 2.671; p = .008) predicted the students’ self-efficacy

144
146

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

and therefore possible proficiency in writing. It was evident that exposing journalism students
to scholarly media developed their self-efficacy towards producing professional news.
Some studies on technology-focused efficacy indicate a positive relationship between students’technology exposure and usage and their self-efficacy for technology in different academic andprofessional contexts. Shank and Cotton’s (2014) study indicated that students could
vicariously acquire technology self-efficacy from their teachers, implying that educator
characteristics could inculcate beliefs in students’ confident use of technologies. Joo & Choi(2000) explored how students’ Internet self-efficacy related to their Internet research
performance based on written and practical tests. Students’ Internet self-efficacy was related
more to the practical research than to the theoretical test. This was not surprising since handson skills are considered the best test of readiness for work.
Research Context
In Rwanda, periodic media industry surveys have partly attributed the low quality of news
content to poor use of online tools by practitioners. A 2017 Rwanda Governance Board survey
considered poor content as the bane of an industry struggling to stay afloat. About 90% of
media houses’ staff surveyed were confirmed to have inadequate skills for producing
professional stories. Although Rwanda’s media policy (2014-2020) envisaged a citizenry thatis exposed to news through a digitally-empowered media, the 2017 National Media Dialogueechoed a growing stakeholder concern that portrays lackluster use of online tools by media
houses (Mwai, 2017). Journalism schools have largely been criticized for inadequate online
skills development for the industry (Media High Council, 2016). While media training
institutions have increased efforts to train the next generation of “digital journalists”,
stakeholders are still concerned about the nature of online skills taught and techniques
employed (Media High Council, 2016). Although students’ efficacy beliefs for technology use
have been explored in other countries, the role of Rwanda’s journalism training on students’
self-efficacy beliefs for professional online skills is not known. Given that employers expect“online-ready” graduates, the role of such training becomes pertinent.
To understand the role of training factors in the students’ self-efficacy for online journalism
work, the role of content taught, training approaches and training facilities were explored. The
study questions were:
1. How efficacious are Rwandan mass communication students in their online journalism
skills?
2. How does the content taught, training approaches and facilities influence the students’
online journalism self-efficacy beliefs?
3. Are there significant relationships between the training factors and the students’ online
journalism self-efficacy?
The null hypothesis (H0) posited no significant positive relationship between each of the
training factors and the students’ online journalism self-efficacy.
Methods
This study employed a descriptive cross-sectional survey involving a sample of final
undergraduate mass communication students (n=182) drawn from journalism schools in five
Rwandan universities. Yamane’s (1967) formula was used to determine the sample size (i.e.

145
147

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

for populations of 1500). This sample comprised 98 males and 84 females and together
constituted 62% of the total student journalism and communication student population(n=293). Recruitment of participants was done through a systematic random sampling fromclass lists provided by the heads of the selected schools. Out of the 182 respondents, 143 (79%)
successfully completed and returned the questionnaires.
The survey instrument was developed through an extensive literature review to identify a scale
incorporating components for the training factors and online journalism self-efficacy. Ferrari’s(2012) digital competence framework was adapted to test the students’ self-efficacy beliefs intheir ability to perform the following five online journalism skills: conduct online research;communicate effectively with different social media tools; develop and share multimedia
content; integrate ethical practices in online news publishing; and use social media tools tosolve organizational problems. The respondents indicated their agreement with statements
about training content, teaching style and training facilities and the five online journalism
skills. This was indicated on a five-point Likert-type scale where 1 = Strongly Disagree and 5= Strongly Agree.
To test the quality of the instrument, a pilot survey involving 20 randomly selected students(i.e. four from each journalism school) preceded the main study. The Cronbach Alpha test
indicated an overall instrument reliability score of .79, with factor-wise reliability values
ranging from .072 to .81. Having met the threshold of .07 as recommended (see Faizan andZehra, 2016), the instrument was found fit for the main study.
The gathered data was analysed using SPSS version 21. Descriptive analyses of the responses
showed the patterns of agreements and disagreements with given statements. Inferential
statistics (correlation and regression analyses) were carried out to show the nature andcontribution of the training factors (independent variables) on the students’ efficacy for online
journalism work (dependent variable). Using F-test, the H0 was tested to confirm or disconfirm
the relationship between training factors and students’ efficacy for online journalism. The
hypothesis was tested at a level of significance of 0.05. Results were then used to draw relevant
conclusions of the study.
The research was part of a broader academic study approved by Rwanda’s National Council
for Science and Technology after ethical clearance by the Directorate of Research of the
University of Rwanda. All respondents were informed about their participation rights and theirconsent obtained before commencement. All responses were kept confidential with no identity
required on the questionnaires returned.
Results
Descriptive Analysis of Online Journalism Training Factors
Drawing from empirical research, the training factors considered critical in the development of
online journalism skills included the nature of online skills taught, training style and training
facilities. Mean aggregate responses regarding these factors are summarized in Table 1.
Results showed that having learnt multimedia content production and sharing skills had thehighest mean score (mean=4.11), followed by learning of social media skills (mean=3.99). The
fact that most of the journalism modules taught integrated notions of online skills (e.g.
research) attracted a mean score of 3.69. Furthermore, respondents indicated that content that
related to the industry (e.g. case studies) were well employed in the online journalism classes

146
148

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

(mean=3.67). These results implied that the respondents had been taught a broad range ofindustry-focused online skills. From the aggregate mean agreement score of 3.87 and standard
deviation of 1.09, respondents generally agreed with the training content despite some slightvariances in their responses.
Table 1: Means and STDVs of training factor responses
Construct
Training
content

Indicator
We learnt some online skills in most of the modules

Mean
3.69

STDV
1.077

We learnt how to use different social media tools
We learnt how to produce and share multimedia content online
We learnt real online case studies from the industry
Sub-variable Aggregate score
Training
Teaching online skills was more practical than theoretical
approach
Online resources like tutorials were sometimes used in teachingSome students volunteered to teach complex online skills like
web design
Lecturers used online tools to professionally interact withstudents
Lecturers demonstrated adequate online journalism skills
Sub-variable Aggregate score
Training
The training facilities (software and hardware) were adequate
facilities
Mobile phones were sometimes used in multimedia production
exercises
Internet connectivity was reliable during online-based classes
Technical support in online journalism classes was always
available
Sub-variable Aggregate score

3.99
4.11
3.67
3.87
4.10

1.051
1.035
1.179
1.09
1.115

3.92
3.73

1.015
1.181

4.22

0.958

3.92
3.98
3.53

1.101
1.07
1.118

4.13

1.013

3.64
3.60

1.178
1.139

3.73

1.112

Overall aggregate score for training factors

3.9

1.1

Being a key component of online journalism education, the prevalence of multimedia
production content and social media skills was not surprising. Scholars argue that journalism
educators need to take cues from the technological advances in the industry and integrate suchskills in the curricula, leveraging social media interactions with the students for professional
teaching and learning (Aifan, 2015; Mihailidis & Shumow, 2011). The results concur withsuggestions for retooling training content in cognizance of most students’ heavy “digital
culture”. By leveraging this digital exposure, scholars argue that trainers would not only savelots of time in teaching already familiar topics but also encourage students to teach themselvesmuch of the emerging content (Ferrucci, 2017; Oluchi, 2016). The high mean score regarding
use of case studies aligned to industry supports researcher recommendations on updating
journalism curricula and employing social media tools in simulated newsrooms for news
production (Iyer, 2015; Hirst & Treadwell, 2011).

147
149

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Regarding the approaches to online journalism training, the online interactions between
lecturers and students (e.g. through email and social networking sites) had the highest mean
agreement score (M=4.22, STDV=0.958). This was followed by the lecturers’ use of morepractical than theoretical training approaches like online projects and case studies. (M=4.10,
STDV=1.115) and then the use of free online tutorials to supplement formal teaching ofmultimedia skills like audio or video editing (M=3.92, STDV=1.015). Respondents also
generally considered the trainers as competent and therefore, able to teach the online skills(M=3.92, STDV=1.101). While seemingly recognizing that a number of students were digitalsavvy and therefore could have learnt a number of online skills by themselves, the voluntary
contribution of students in teaching complex topics like web design and development was
acknowledged by most of the respondents (see M=3.73, STDV=1.181).
The results align with studies that suggest the need for journalism educators to adopt pedagogy
strategies that impart the required digital skills for the ever-changing industry. Wotkyns’(2014) study on the level of students’ satisfaction with journalism curricula particularly
highlighted their appreciation with experiential learning approaches that integrated theircreativity and helped to link theory with practice. Jeanti (2015) and Ferrucci (2017) also argue
that teaching digital journalism should be a process where the trainer co-designs the training
with his or her students, allowing for as much innovation as possible. Incorporating students inthe teaching of online skills is also touted as relief for trainers since they are not necessarily
expected to be well-versed with all emerging online tools (Schwalbe, 2009). Results from thecurrent study therefore seem to support this approach where students are taught to largely drive
their own learning through the vast number of online tools.
From the results, mobile phone use in multimedia production training has gained ground in thejournalism schools, with a mean agreement score of 4.13 implying that phones were being used
to supplement the more traditional tools of news production like recorders, cameras andcomputers. The low mean score regarding the adequacy of these traditional training facilities(M=3.53, STDV=1.118) possibly indicated that in the absence of these facilities, the easily
accessible mobile phones came in handy for practical learning purposes. Online journalism
training also seemed to by the availability of reliable internet connection (M=3.64,
STDV=1.178) which is the bedrock of online skills production. To ensure online journalismclasses ran well even in the absence of the lecturers’ interventions, respondents generally
agreed that technical assistance was always available (M=3.60, STDV=1.139). This impliedthat apart from technical assistance from knowledgeable student colleagues, it would seem thatthe training facilities were manned with support staff not only to ensure the facilities wereready for use but also to support students with the online production projects when necessary.
The results concur with other scholarly views on the need for journalism training institutions
to appropriate new media tools that will enhance the online story-telling practices of the future
professionals (Aifan, 2015; Bor, 2014; Wenger & Owen, 2012). As the results show, easy
access to mobile phones as tools of journalistic production can address costs associated withthe traditional journalism equipment (Ferruci, 2017; Salaverría, 2011). Although the ideal
online journalism training facilities are far from being realized (e.g. for reasons of limited
financial means), studies advocate for connected multimedia environments that afford students
with unlimited opportunities to experiment and practice their new media production skills(Alves et al., 2014; Switzer & Switzer, 2013).
Overall, constructs under the training approach had the highest aggregate mean agreementscore among the respondents (M=3.98, STDV=1.07). This was followed by constructs under

148
150

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

training content (M=3.87, STDV=1.09) and then constructs under training facilities (M=3.73,
STDV=1.11). The aggregate mean score of all the training factors (M=3.9) indicated that most
respondents generally agreed about the different aspects of the training in online journalism.
The impression created by these results was that students agreed (with moderate variability)that they were trained on a variety of relevant online skills, using innovative techniques which
possibly complemented the traditional teaching approaches and modest online training
facilities. This demonstrates the dynamic nature of online journalism training where scholars
advocate for a keen eye on the evolving digital needs of the media industry coupled withrevised curricula, innovative training and learning techniques and general technical readinessfor a future of technology that is gradually redefining the roles of journalists (Robinson, 2013;
Jeanti, 2015; Tanner, 2014).
Descriptive Analysis of Online Journalism Efficacy
The respondents’ self-efficacy for online journalism was conceptualized as their beliefs in theability to effectively do online journalism research, create and share multimedia stories online,
use social media tools to communicate to different audiences, practice ethical online publishingand solve problems with different online tools. Table 2 shows the aggregated responses under
this variable.
Table 2: Respondents agreement levels on their online journalism efficacy
I believe I can effectively…

Generally Agree(SA+A)
84.4
76.7
73.7

Neutral
(N)
12.1
12.9
15.0

Generally Disagree(D+SD)
3.5
10.3
11.3

Conduct online journalism researchCommunicate with different social media
Create and share multimedia stories
onlineConduct ethical online publishing
80.4
15.0
4.5
Use online tools to solve different
85.3
9.9
4.8
problems
AVERAGE
80.1
13
6.9
Key: SA=Strongly Agree, A=Agree, N=Neutral, D=Disagree, SD=Strongly Disagree

From the results, respondents considered themselves adept in the five online journalism skills,
with an average of more than 80% of them believing they could perform all the skills. The
greatest level of self-efficacy for online journalism was expressed in the ability to use online
tools to solve problems (85.3%), followed by ability to conduct research online (84.4%) andunderstanding of the ethical implications of using online tools (80.4%). Compared to otherdimensions, the respondents had low efficacy for multimedia production skills (73.7%). Only
about 7% of the respondents did not express any efficacy to execute online journalism tasks.
The findings are consistent with studies that explored the digital competence of students. For
example, Sutherland and Ho (2017) and Bethell (2010), argued that digital competence should
best be demonstrated by the ability to identify and solve practical problems. Although scholars
like Switzer & Switzer (2013) and Bor (2014) suggest that students are likely to gain social
media communications skills faster than other skills such as multimedia and problem-solving
skills, this study showed the contrary. It seemed to indicate that the students focused more on
interrogating how social media tools can be used to address problems in organization as well

149
151

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

as online research skills than others. To obtain a fuller picture on the relationship between theidentified training factors and the students’ levels of online journalism efficacy, the next section
explores the statistical nature of the relationships as well as contribution of each of the factors
on the students’ online journalism self-efficacy levels.
Correlation Analysis of Training Factors and Online Journalism Efficacy Beliefs
To determine the strength and direction of association between the training factors and thestudents’ self-efficacy for online journalism, correlation analysis was done. The resulting
correlation matrix with correlation coefficients for the aggregate of the training factors andonline journalism efficacy is shown on Table 3.
Table 3: Correlations of training factors and online journalism efficacy
Training factor

Training
content

Trainingstyle

Training
Facilities

Online journalism
self-efficacy

Training contentr1
Sig. (2-tailed)
Training styler.413**
1
Sig. (2-tailed).000
Training Facilities r.489**.419**
1
Sig. (2-tailed)
.000
.000
**
Online journalism r.435.315**.422**
1
self-efficacy
Sig. (2-tailed)
.000
.000
.000
**. Correlation is significant at the 0.01 level (2-tailed); r = Pearson correlation coefficient
Results showed that all the training factors were significant and positively correlated to theonline journalism self-efficacy levels of the students. The nature and types of online skills
taught had the highest significant positive correlation with the students’ online journalism selfefficacy (r=.435, p=.000) followed by training facilities (r=.422, p=0.000). The way online
skills were taught had the least yet significant relationship with the students’ online journalism
self-efficacy (r=.315, p=.000) at 95% level of confidence. The results implied that a unit ofpositive improvement in the nature and types of online skills taught, training facilities andtraining approaches led to a corresponding increase in the students’ online journalism selfefficacy levels by 43.5%, 42.2% and 31.5% respectively. Further, the results disconfirmed theH0 that predicted no significant correlation between training factors and online journalism
efficacy. The observed positive linear relationships supported studies that underscore the needfor journalism schools to adapt their training to the ever-changing media industry if future
professionals were to fit well in the digital industry (Ferrucci, 2017, Ferruci, 2017; Jeanti,2015).
Test of Hypotheses
To explore the statistical significance of the influence of the training factors on the students’
self-efficacy for online journalism, the null hypothesis postulated no significant relationshipbetween the training factors and the online journalism self-efficacy. To test this hypothesis, amultiple regression analysis model was employed to establish if an aggregate mean score of
the training factors could statistically predict the students’ online journalism self-efficacy at
95% level of confidence. The postulated model was fitted thus: Y = β0+ β1X1+ β2X2 +β3X3+Ɛi,
where Y = online journalism self-efficacy, β0 = constant (α = constant term), β1- β3= intercepts

150
152

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

for the independent variables, X1= training content or online skills taught, X2= training
approaches, X3= training facilities and ε = error term/Stochastic term.
The results in Table 4 presents the fitness of the regression model used in explaining thevariation of online journalism self-efficacy as a result of the identified training factors.
Table 4: Model Summary for training factors on students’ online journalism self-efficacy
Model

1

R

R2

.545a .297

Adjuste
d R2
.282

Std.
Error ofthe
Estimate.474

R Change

Change Statistics
F
df1
df2
ChangeSig. F
Change.297

19.288

.000

2

3

137

a. Predictors: (Constant), Training facilities, Training style, Training content
Results showed that, taken together, all the training factors were highly and significantly
correlated with the students’ online journalism self-efficacy (r=.545, p=.000). Furthermore,
given the coefficient of determination of 0.297 (R2), it was evident that the training factors
exerted some explanatory power on the students’ online journalism self-efficacy. This impliedthat 29.7% of variations in the online journalism efficacy could be explained by the training
factors (on their own in the model). The remaining 70.3% could only be explained by issues
not factored in the model. However, without the constant variable (β0) on the model, thetraining factors had a predictive power of 28.20% (adjusted R2 of .282) on the students’ online
journalism self-efficacy, implying only a minor variation of .015. This adjusted R2 result meantthat a unit improvement of the training factors (without the β0) would improve the students’
self-efficacy for online journalism by a factor of .282 or 28%.
The Analysis of Variance (ANOVA) results (Table 5) indicated that the proposed regression
model had a statistically significant goodness-of-fit at 5% significance, indicated by thecalculated F-statistic of 19.288 which was larger than the critical F-value of 2.67 (obtained
from the F distribution tables) with degrees of freedom (3,140) at p-value<0.000. Moreover,the existence of a significant positive relationship between the training factors and online
journalism efficacy of the students implied that the proposed model could be relied upon todemonstrate the predictive power of training factors on the online journalism efficacy of thestudents.
Table 5: ANOVAa for training factors on students’ online journalism self-efficacy beliefs
Model

Sum ofdf
Mean SquareF
Squares
1 Regression
13.0073
4.336
19.288Residual
30.795
140
.225
Total
43.801
143
a. Dependent Variable: Online journalism self-efficacy
b. Predictors: (Constant), Training Facilities, Training style, Training content

Sig.
.000b

By further testing the significance of the regression coefficients of the training factors on thestudents’ online journalism efficacy (see Table 6), results confirmed a positive relationship

151
153

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

between the training content (β=0.059, t=2.780) and training facilities (β=0.076, t=4.058) andthe students’ online journalism self-efficacy, with a significance of .006 and .000 respectively.
Despite the training approaches having indicated a positive relationship with the students’
online journalism efficacy, this relationship was of negligible significance (p>.349)
Table 6: Coefficients for training factorsa
Model

UnstandardizedStandardized
Coefficients
Coefficients
B
Std. Error
Beta
1 (Constant)
2.191
.271
Training content/skills
.059
.021
.236
Training approaches
.018
.019
.077
Training facilities
.076
.019
.346
a. Dependent Variable: Online journalism self-efficacy

t

8.090
2.780
.940
4.058

Sig.

.000
.006
.349
.000

From the results, the proposed model predicting online journalism efficacy from the three
training factors taken together was, therefore, fitted with regression coefficients as follows:
online journalism self-efficacy = 2.191 + 0.059 (training content) + 0.018 (training approaches)+ 0.076 (training facilities) + Ɛ.
With p-values 0.000<0.05 for both training content and training facilities, the H0 was therefore
rejected and a conclusion made that these training factors exerted a significant and positive
influence on the online journalism self-efficacy beliefs of mass communication students in
Rwanda. In an optimal model, however, the H0 for training approaches would be accepted since
it had a p-value>.05, indicating that, though this dimension had a positive relationship withstudents’ self-efficacy for online journalism, its significance did not meet the 95% level ofconfidence. These findings agree with scholarly evidence that advocate for adapting thejournalism training to the required digital content, training approaches and facilities to improvethe career paths of graduates in the digitally-oriented industry (McDevitt & Sindorf, 2012;
Cindy, 2015).
Discussion of Findings
This study sought to explore the relationship between training factors and online journalism
self-efficacy beliefs of mass communication students in Rwanda. The results largely supported
scholarly evidence that training determines the perceived work-preparedness of students.
Respondents generally agreed with the different statements regarding the characteristics ofonline skills offered, the approaches used in teaching these skills as well as the nature offacilities available for teaching and learning online journalism skills. The fact that a good mixof multimedia production and social media skills are well integrated in the curriculum pointed
to alignment of curricula with emerging digital skills in the industry. Of note also were the highmean scores regarding the online skills or content taught as well as the training facilities. This
indicated that Rwandan journalism schools seem to have made appreciable strides in adapting
online tools in the curricula and invested in some facilities to enhance the teaching and learningof these emerging skills.
Correlation analyses showed significant positive correlations between each of the training
factors and online journalism self-efficacy beliefs of the students. This demonstrated that

152
154

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

journalism schools cannot ignore the role played by characteristics of training in their quest to
develop online-ready graduates. Media researchers have argued for competence-based training
characterized by teaching skills or content that will make graduates fit well in the workplace(Hirst & Treadwell, 2011). In teaching online skills among the youth such as students, evidenceshows that pedagogy which considers the digital habits of these digital natives might facilitatecompetence development of digital skills which are transferable to the industry (Jeanti, 2015).
In this study, respondents seemed to recognize that some of their colleagues had more advanced
online skills than the lecturers which enabled them to contribute in the teaching and learningof more complex practical exercises like web design and development. This supports
arguments by scholars that in this era of new media technologies, journalism educators should
embrace co-creation of training content and innovative ideas from the learners (Wiebe &McAuley, 2010).
In the context of online journalism education, digital training resources have been considered
key as the rapid development of new technologies requires frequent updates of theinfrastructure (Boers et al., 2012). In this study, though technical facilities had a weak but
positive correlation with the online journalism self-efficacy beliefs of the students, the highmean score regarding the use of mobile phones and online tutorials in the multimedia
production classes gives hope especially with dwindling resources for the more expensive
technical journalism equipment. This is in line with arguments from scholars like Mihailidis
and Shumow (2011) who decry the way some journalism schools compromise practicallearning through “bureaucratic inertia and resource constraints” (p.15) instead of opting forcheaper and more cost-effective alternatives like mobile phones.
The respondents’ good rating on the accessibility to reliable Internet connection and technicalsupport during the training also indicated attempts at technical readiness for the schools
concerned. This augurs well for online skills training since researchers argue that training in
new media technologies require a good mix of tools and techniques all of which depend on theavailability of a reliable source of Internet connectivity (Iyer, 2015; Jeanti, 2015).
By showing that the aggregate mean score of training factors had significant contribution(R2=.297, p<0.05) in the students’ self-efficacy beliefs regarding online journalism work, theregression analysis results demonstrated the central role of these characteristics of training in
explaining or predicting the level of the “online-readiness” of the mass communication
students. Particularly, disaggregated mean scores also indicated that each of the factors had
contribution though with different levels of significance. The findings resonate with studies
that position the revival of journalism curricula and pedagogy techniques as central to theintegration of new media skills in journalism training. Such studies argue for the need forjournalism schools to identify the range of skills or content to be taught and how these will be
continuously and effectively taught in an increasingly dynamic media industry (Tanner, 2014;
Ferrucci, 2017).
Conclusion
The study explored perceptions of mass communication students regarding online skills
training and how these are congruent with their online journalism self-efficacy beliefs. The
respondents expressed agreement in all the factors as characteristic of the online journalism
training in Rwandan journalism schools. This portends well for the development of digitalskills among the future media professionals. In line with previous research, results showed thatthe content taught, how it is taught and the overall training environment were correlated with

153
155

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

the students’ self-efficacy for online journalism (Cindy, 2015; Seelig, 2010). In essence, this
implied that improving the quality and diversity of online skills taught and how they aredelivered builds confidence in the students’ ability to use online tools to work in the industry.
Regression analysis also confirmed that the training factors (singly and collectively) positively
contribute to the students’ levels of online journalism self-efficacy, though at varying levels of
statistical significance.
These results are encouraging for online journalism education in Rwanda. However, the study
had some limitations that merit further investigations. Given the narrow focus of the study,
including a combination of other factors beyond training (like field experiences and individual
online behavior) might have produced more insightful results on the self-efficacy of thestudents. Furthermore, the study was limited by the use of a survey. The nuances of the lived
experiences of a phenomenon (in this case, how students make sense of the factors incidental
to their self-efficacy beliefs in online journalism) are best explored with a mixed methods
approach. Including interviews and/or focus group discussions with selected students might
possibly yield a clearer picture of the students’ online journalism self-efficacy beliefs. Despite
these limitations, this study can be considered as an addition to knowledge on how the
dynamics of online journalism training in the digital era are likely to play into building thenecessary confidence for students as they face the increasingly digitally-competitive
workplace.

154
156

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Aifan, H. A. (2015). Saudi students’ attitudes toward using social media to support learning.
(Unpublished PhD thesis), University of Kansas.
Alves, K. C., de Souza Filho, G., Moura S. and Brito F. (2014). Collaborative learning indigital journalism: Using JCollab for journalists’ education. Brazilian Journalism
Research 10(1), 238–259. https://doi.org/10.25200/BJR.v10n1.2014.633
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, Prentice-Hall.
Becker, L., Vlad, T and Kalpen, K. (2012). Annual survey of mass communication
enrolments: enrolments decline for second year in a row. Journalism & Mass
Communication Educator, 67(4), 333–361.
https://doi.org/10.1177/1077695814555432.
Bethell, P (2010). Journalism students’ experience of mobile phone technology: implications
for journalism education. Asia Pacific Media Educator, 20, 103–114.
Boers, R., Ercan, E., Rinsdorf, L. and Vaagan, R. (2012). From convergence to connectivism:
Teaching Journalism 2.0. Online Journal of Communication and Media Technologies,
2(4), 52–64. https://doi.org/10.29333/ojcmt/2398.
Bor S. E. (2014) Teaching social media journalism: Challenges and opportunities for future
curriculum design. Journalism & Mass Communication Educator, 69, 243–255.
https://doi.org/10.1177/1077695814531767.
Broaddus, M. B. (2012). Students’ writing self-efficacy, motivation, and experience:
Predictors in journalism education. (Unpublished PhD dissertation), University ofTennessee.
Ehreneich, B. (2009). Welcome to a dying industry, journalism grads. SFGate. Retrieved
from http://www.sfgate.com/cgi-bin/article.cgi.
Faizan, R. and Zehra, N. (2016). Quality work-life as predictor to organisational commitment
under contrasting leadership styles: I.T responses from Pakistan's private softwarehouses. Global Journal of Management and Administration, 16(6), 9–23.
Ferrari, A. (2012). Digital competence in practice: An analysis of frameworks. JRC
Technical Reports. Institute for Prospective Technological Studies.
Ferrucci, P (2017). “We’ve lost the basics”: Perceptions of journalism education fromveterans in the field. Journalism & Mass Communication Educator, 1–11.
https://doi.org/10.1177/1077695817731870.
Gallardo-Echenique E., Minelli de Oliveira J., Marqués-Molias L., Esteve-Mon F. (2015).
Digital competence in the knowledge society. MERLOT Journal of Online Learning
and Teaching, 11(1), 1–16.
Hirst, M and Treadwell G (2011). Blogs bother me: Social media, journalism students and thecurriculum. Journalism Practice 5, 446–461.
https://doi.org/10.1080/17512786.2011.555367.
Huang, K.T., Ball, C., Cotton, S. R. & O’Neal, L. (2020). Effective experiences: A social
cognitive analysis of young students’ technology self-efficacy and STEM attitudes.
Social Inclusion, 8(2), 213–221. https://doi.org/10.17645/si.v8i2.2612.

155
157

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Iordache, C., Mariën, I., & Baelden, D. (2017). Developing digital skills and competences: A
quick-scan analysis of 13 digital literacy models. Italian Journal of Sociology of
Education, 9(1), 6–30. https://doi.org/10.14658/pupj-ijse-2017-1-2.
Iyer, P. (2015). Competencies for future newsrooms in Australia: a mid-career learning
strategy for journalists. (Unpublished PhD Thesis), University of Wollongong,
Australia.
Jeanti, S. (2015). Doing it for real: Designing experiential journalism curricula that prepare
Students for the new and uncertain world of journalism work. Coolabah 16, 122–142.
https://doi.org/10.1344/co201516122-142.
Joo, Y. & Choi, M.B. (2000). Self-efficacy for self-regulated learning, academic selfefficacy, and Internet self-efficacy in Web-based instruction. Educational Technology
Research and Development, 48(2):5–17. https://doi.org/10.1007/BF02313398.
Kaul, V. (2013). Journalism in the age of digital technology. Online Journal ofCommunication and Media Technologies, 3(1), 125–143.
https://doi.org/10.29333/ojcmt/2414
McDevitt, M. & Sindorf, S. (2012). How to kill a journalism school: The digital sublime inthe discourse of discontinuance. Journalism & Mass Communication Educator, 67,
109–118. https//doi.org/10.1177/1077695812440942.
Media High Council (2016). Training report on web design and maintenance for Rwandan
journalists. Kigali, Rwanda.
Mihailidis, P and Shumow, M (2011). Theorizing journalism education, citizenship, and new
media technologies in a global media age. Taiwan Journal of Democracy, 7(2), 27–
47. https://doi.org/10.29654/TJD.201112.0003.
Mwai, C (2017, November 7). Local media mulls survival in digital age. The New Times.
https://www.newtimes.co.rw/section/read/223157
Oluchi, E. (2016). Media Convergence: A Paradigm Shift in Journalism Education in
Nigeria. Review of Journalism and Mass Communication, 4(1), 19–42.
https://doi.org/10.15640/rjmc.v4n1a2.
Patrão, C. & Figueiredo, A. (2018). A social web based pedagogical strategy for teaching
journalism in higher education. RED-Revista de Educación a Distancia [Distance
Education Magazine], 57(8). https:// doi.org/10.6018/red/57/8.
Quinn, S (2010). Opportunities for journalism education in an online entrepreneurial world.
Asia Pacific Media Educator, 20, 69–80.
Robinson, S (2013). Teaching journalism as process: A proposed paradigm for J-School
curricula in the digital age. Teaching Journalism and Mass Communication, 3(1), 1–
12.
Royal, C. (2015) Developing and assessing experiential learning opportunities. Teaching
Journalism and Mass Communication, 5(1), 22–32.
Rwanda Governance Board (2018). Rwanda Media Barometer (2018). Kigali, Rwanda.
Salaverría, R. (2011). Online journalism meets the university: Ideas for teaching andresearch. Brazilian Journalism Research, 7(2), 137–152.
https://doi.org/10.25200/BJR.v7n2.2011.356

156
158

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Schwalbe, C. B. (2009). Leveraging the digital media habits of the millennials: Strategies forteaching journalism courses. Southwestern Mass Communication Journal, 25, 53–68.
Seelig, M. (2010). Journalism and mass communication education: The impact of technology
on pedagogy. Explorations in Media Ecology, 9 (4), 245–258.
Shank, D. B. & Cotton, S. R. (2014). Does technology empower urban youth? The
relationship of technology use to self-efficacy. Computers & Education, 70, 184–193.
https://doi.org/10.1016/j.compedu.2013.08.018.
Sutherland, K and Ho, S, (2017). Undergraduate perceptions of social media proficiency andgraduate employability: A pilot study. Higher Education, Skills and Work-Based
Learning, 7(3), 261–274. https://doi.org/10.1108/HESWBL-02-2017-0018.
Switzer, J & Switzer, R (2013). The myth of the tech-savvy student: the role of media
educators in a Web 2.0. World. Journal of Media Education, 4(4). 15–27.
Tanner, S. (2014). Graduate qualities and journalism curriculum renewal: Balancing tertiary
expectations and industry needs in a changing environment. (Unpublished PhD
thesis), Edith Cowan University, Australia.
Wenger, D., & Owens, L. (2012). Help wanted 2010: An examination of new media skills
required by top U.S. news companies. Journalism and Mass Communication
Educator, 67(1), 9–25. https://doi.org/10.1177/1077695811426826
Wiebe, S. & Mcauley, S. (2010). Harnessing new technologies to teach academic writing to
the Net generation. in education, 16 (1), 117–130.
Wotkyns III, R. S. (2014). Curriculum development in the digital age of journalism. GSTF
Journal on Media & Communications, 2(1), 28–33. https://doi.org/10.7603/s40874014-0005-6.
Yamane, T. (1967). Statistics: An introductory analysis, (2nd ed.). Harper and Row.
Corresponding author: Joseph Njuguna
Contact e-mail: jnjuguna110@gmail.com

157
159

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

158
160

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Developing an Open Educational Resource and Exploring OER-Enabled Pedagogy
in Higher Education
Beth Tillinghast
University of Hawaii at Manoa
United States

159
161

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Abstract
There is a growing trend in higher education to explore the various benefits of Open
Educational Resources. This applies not only to the cost benefits, but also to potential
pedagogical benefits as well. This study explores the process of developing and implementing
an Open Educational Resource for an undergraduate course and experimenting with OERenabled pedagogy. Interview data provide an account of this process, outlining challenges andhighlighting insights, which might prove useful for other professionals contemplating the movetoward developing Open Educational Resources. Interview data are also organized using theUnified Theory of Acceptance and Use of Technology constructs pertaining to attitude,
performance expectancy, effort expectancy, social influence, technology self-efficacy, andfacilitating conditions. A qualitative interpretive approach was then used to analyze the data.
Analysis indicates that most of the constructs can strongly influence faculty to adopt Open
Educational Resources but that “social influence” has no effect on adoption. Findings also
include reflections on OER-enabled pedagogy as it was applied in the course. These findings
indicate that OER-enabled pedagogy has the potential for increasing student engagement,
though this potential has not yet been realized. Finally, findings provide an outline ofrecommendations that might guide others when considering developing and implementing
Open Educational Resources.
Keywords: Open Educational Resources, OER-enabled pedagogy, UTAUT; OER, OER use,development

160
162

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Over the centuries, the concept of formal education has been intricately coupled with the ideathat books are an indispensable component of that experience (Reed, 2018). Indeed, some of
the first institutions of learning were formed around large collections of materials such as foundat the Library of Alexandria (Cubberley, 1902). Modern education has continued todemonstrate this reliance on books, with an ongoing dependence on commercial textbooks as
an integral part of the curriculum (Weller, 2014). Unfortunately, this practice has been shownto have an impact on the student experience in the current higher education context. This is dueto the high cost of textbooks, which can potentially prove to be an access barrier to essential
educational resources (Brandle et al., 2019). In fact, as of 2017, the U.S. Bureau of Labor
Statistics reported that the cost of textbooks rose by 142% over the last decade and a half,
which represents a rate that is four times that of inflation (U.S. Bureau of Labor Statistics,
2017). As a result of these high costs, students may choose not to purchase textbooks. In a
study published in 2018 by Wakefield Research, 42% of students surveyed said they had
avoided purchasing course materials because of costs (Lederman, 2018).
Considering this backdrop, it is no wonder that there has been an uptake in interest in the
adoption and development of Open Educational Resources (OER). OER are learning resourcesthat are made available either at no cost in digital format or for very low cost in print format.
They either reside in the public domain or carry with them the necessary Creative Commonslicenses to allow for flexible reuse, revision, remixing, retention, and redistribution (the 5 Rs)
increasing their flexibility (Wiley et al., 2014). It becomes apparent that OER, providing bothfree and flexible access, would be a logical alternative in addressing the textbook cost issue. In
addition to the cost-saving aspect of OER, others are exploring the potential to promote open
practices and to investigate pedagogical shifts through the use of these resources (DeRosa &Robinson, 2017; Ehlers, 2011; Wiley & Hilton III, 2018).
This research examines the experiences and perceptions of faculty and staff as they develop an
OER for an undergraduate nutrition course and apply it in instruction. It further examines theexperience of one instructor undertaking teaching with OER-enabled pedagogy (OP), which isa pedagogical approach involving students in the co-creation of content (Wiley & Hilton III,2018). Findings are organized by recording the experiences of faculty and staff as they develop
and implement an OER and then by examining perceptions through the lens of the Unified
Theory of Acceptance and Use of Technology (UTAUT) model. Recommendations from thisexperience are also presented, which might prove useful for exploring the development of
OER. This knowledge could, in turn, be used to promote greater use and development of OER,
and, therefore, to support greater student access to and application of learning resources. This
access might subsequently impact student success in higher education.
Literature Review
A large amount of research exists that explores factors that might motivate faculty to adopt
OER and factors that might present challenges (Algers & Silva-Fletcher, 2015; Anderson etal., 2017). The literature shows that faculty are motivated to adopt OER in order to cut costs
for students and to enhance educational equity (Belikov & Bodily, 2016) as well as to pursue
pedagogical freedom (Dermody, 2019).
A more recent movement in the consideration of OER adoption is how these resources play arole in the further development of open educational practices (OEP) and how these practices
can potentially enhance student learning (Cronin, 2017; see Figure 1).

161
163

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Figure 1: OER can play a role in promoting various aspects of OEP.
One of the first discussions of OEP was put forth by Ehlers (2011), who described a second
phase of OER development whereby there is a shift from a focus on OER as merely resources
to a focus on how they can promote open practices in education. In a paper discussing how
open resources support the exploration of open pedagogy, DeRosa and Robinson (2017)
reported that when students are exposed to and engaged in the use and reuse of learning
resources, they can begin to develop new and deeper relationships with the resources, whichmight impact learning. Since the term “open pedagogy” has become associated with OER in
recent years, and in order to avoid confusion, this research used the term “OER-enabledpedagogy” as defined by Wiley and Hilton (2018) as “the set of teaching and learning practices
that are only possible or practical in the context of the 5R permissions that are characteristic of
OER” (p. 133).
The scholarly literature does not provide in-depth research on faculty who have alreadyadopted OER and who may be experimenting with OP. In addition, it does not provide a great
deal of research detailing the process of OER creation and application within a course(Mallinson & Krull, 2015). An account of this process might prove useful for otherprofessionals contemplating the move toward OER creation by providing insights into theactual process as well as useful guideposts along the way. Therefore, the focus of this research
was to explore the experiences and perceptions of faculty and staff in higher education as they
initiated and developed an OER for an undergraduate course and then applied it in instruction.
Methodology
This study applied an interpretive qualitative approach (Creswell et al., 2006) in order toexplore the perspectives of faculty and staff engaged with the development and application of
an OER textbook and OP in the context of higher education. This approach uses word-based
data in its analysis and views human action as meaningful and best studied in its social contexts(Yanow & Schwartz-Shea, 2006). Eight participants took part in this research. They
represented the various roles involved with the development and application of an OER for one
specific undergraduate class at a research-intensive university. An exempt status IRB approval
was secured for the study as well as consent for all interviews. Data were recorded, transcribed,
validated by participants, and anonymized.

162
164

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Participants
Eight individuals made up the participating team. Two faculty members had been involved insecuring a university OER Initiative Grant (http://oer.hawaii.edu/projects/) in 2017 and inorganizing and guiding the overall development of a textbook for their academic area. Graduate
assistants (GAs), acting as resource developers, worked on adapting or creating detailed
information within the text under the guidance of the faculty. An institutional OER technologist
provided instructional design and resource development support as well. Instructors
implemented the OER textbook and provided feedback to enhance later modifications. One ofthree course instructors experimented with implementing OP activities within two sections of
the course. Interviews with these participants represented all but one member of the entire teamand represented all of the participant roles.
The UTAUT Model
Constructs associated with the UTAUT framework guided this research (Venkatesh et al.,2003). The UTAUT model is based on eight different technology acceptance models and isapplied in research to explain user intentions to use an information system and subsequent
usage behavior (Venkatesh et al., 2003). This framework is based on core constructs that
determine the intention to adopt a specific technology: performance expectancy, effort
expectancy, social influence, and facilitating conditions. The original UTAUT framework also
included the constructs of attitude and Internet self-efficacy, which were incorporated into theframework for this research.
The UTAUT model has helped to guide research exploring different aspects of technology
acceptance in the context of open access adoption studies and open practices (Dulle & MinishiMajanja, 2011; Kandiero, 2015; Mtebe & Raisamo, 2014; Percy & Van Belle, 2012). Theseauthors have indicated that the original UTAUT framework used in quantitative research has
proved consistent as a model, with high validity and reliability, and is robust in predicting theacceptance of a new technology as compared to other models (Percy & Van Belle, 2012).
Though this model has primarily been applied to research investigating potential acceptanceand use of a technology, it is also appropriate to apply as a guiding framework when exploring
the process of development and adoption of a technology because it examines influencing
factors.
Research Question
The research question explored in this study was as follows:
What are the perceptions and experiences of faculty and staff in the development ofan OER, in instruction introducing the OER, and in the application of OER-enabledpedagogy in an undergraduate course?
Instrumentation
The instrument developed for this study consisted of semi-structured interview questions witha guiding interview protocol. Questions, which were based on the constructs that outline the
UTAUT model, varied as appropriate to the participant role. Questions addressed the roles offaculty, OER technologist, resource developer, and instructor.
The interview questions referenced those developed by a number of researchers applying the
UTAUT model (Dulle and Minsihi-Majana, 2011; Venkatesh et al., 2003). They also
referenced those developed to examine instructional priorities for adopting OER (Jung &

163
165

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Hong, 2016). The referenced instruments were either released under a Creative Commonslicense, which allows reuse, or permission was sought for their use.
Data Collection and Analysis
Qualitative data were gathered through recorded semi-structured interviews with the eightparticipants who were involved throughout the project. Data were then transcribed
professionally and later validated by the interviewees. Analysis was conducted as passages inthe text were identified and coded. Next these codes were categorized and then thematicallyanalyzed. Through this process, the words of the participants were used to deepen
understanding and shed light on the process of OER development and use (Corden &Sainsbury, 2006). The narrative developed from the data was later sent to the participants for
final review and validation.
Findings
Findings for this study were organized in two ways. Firstly, the experiences of the faculty andstaff as they moved from the initial conception of the OER project through the development
and application in instruction were recorded in a narrative format. Secondly, findings and theemerging themes were organized by and recorded through the lens of the UTAUT framework.
Insights from the findings were also translated into recommendations in order to provide a
guide for subsequent OER development.
Development and Application of the OER
Inception. To prepare for the upcoming semester, the lead faculty visited the bookstore to pickup the newest edition of the textbook she had been using. She reported being ‘appalled. It was$130 for a book that didn’t have any binding, no cover, and it had two different sets of page
numbers!’ She didn’t think she could ask students to work with such an obviously “awful”
textbook and immediately emailed the campus OER technologist to find a solution.
The technologist suggested that the faculty apply for a campus OER grant offered through theOutreach College. She applied for and was awarded the grant. At this point, she identified ateam that would become responsible for the development of the OER. The team would
ultimately consist of the lead faculty, a support faculty, the OER Technologist, two graduate
assistants, and three instructors, who would later implement the text.
The goal of the project from the faculty members’ perspective was to develop a free resourcethat could be shared and that was as good as, or better than, the customary commercial
textbook. Also, the faculty wanted a resource over which they had control of the content andone that was “placed-based and more appropriate for their student population”.
Development. The actual work began in January 2017. After the first GA was hired, weekly
meetings became the norm, which continued for the year during the development phase. Thesemeetings became a critical component of the workflow and helped to “ensure a successful
product”. The GA reported that the first order of business was to “brainstorm everything.
Figuring out … what was the mission for this book, what were our goals … then outlining thechapters … before beginning to research”. The technologist relied on his background as an
instructional designer to provide guidance in the overall design of the resource as the team“made a version that was based on how [the faculty] wished the course would be”.

164
166

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

After the initial brainstorming meetings, the faculty members’ role became to first “organize
the sequence of topic areas” and then “find OER resources that were already out there …
judging which ones best fit the level of science that we were teaching and were reliable andseemed to be sound”“. The OER technologist assisted the team in this phase by directing themto existing resources.
Throughout the process, someone needed to take the key role in leading the OER development.
The lead faculty fulfilled this role by “ultimately being the one responsible to see [the project]from start to finish” and to ensure that funding was available.
The GA reported helping with the “basic outline of the textbook, and then actually finding thecontent or writing the content if it wasn’t available”. The GA shared, “The faculty were able
to contribute to [the text] to make sure that we had the key points correct … as a lot of nutrition
books seem to have wrong information”. After the first semester, the Gas changed. The newly
hired GA brought fresh eyes to the project finding that “there were a lot of things missing that
we really wanted to include”. For example, they wanted to make sure that “every chapter there
has a connection to something … that relates to Hawaii or Pacific Islanders”. This GA was also
responsible for finding openly licensed photos and images, which became “a huge, timeconsuming factor”. As the content started emerging, the GA was also aware of the need toprovide a consistent tone and similar structure throughout the material. As the resource more
fully formed, the technologist organized an outside review by working with the Rebus Project,which provides a community of collaborators with publishing guidance for OER projects.
Through this project, the resource was reviewed by “subject matter experts from a couple ofdifferent institutions … and [by] different copy editors … to look for tone and consistency”.
Most of the work had been completed in Google Docs for collaboration purposes. The final
product would be offered as a Pressbook. The OER technologist indicated that “Once you’rein Pressbooks, the process [of developing the OER], the workflow, is not as streamlined, notas smooth”. It was in the Pressbook development phase where the team began “doing the real
aesthetic things of making sure the references looked fine or fixing tables”. The team needed
to work with “the resolution of images so they would print well” and “to add in alternative text
… for accessibility”. The final stage also included sharing the resource beyond the boundariesof the university. The OER technologist “shared it over Twitter, over social media, and emailed
it to a couple of different listservs, and sent it to a few folks … who were waiting for the book”.
Transition into instruction. In the process of transitioning to a new instructional resource, theinstructors reported that there “wasn’t much of an impact [on their workload] … just updating
links on … a personal website”. One instructor reported that the “impact was really positive. It
feels so nice as an instructor to say, ‘We have this free resource for you’”. Another instructor
reported, “a bigger impact [was that] I felt good about the way the information was being shared
to students and the fact that it was free, that they could have access to this textbook even after
the course is over. It is also more relatable, with local information and beautiful local pictures”.
OER-enabled pedagogy. When launching the new textbook, one instructor decided to begin
work on a unit that she felt was missing from nutrition textbooks. Her goal was to begin
developing a unit on “health at every size, body positivity, and size diversity”. Realizing that
this goal would unfold over several semesters, she began by involving students in research inthis field, which would later appear in the textbook. “I was able to create an assignment for the
students [on this topic], and on how to use the library database, and how to look up peer
reviewed journals under [this] topic that were not in the book yet”. She continued by saying,

165
167

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

“The whole premise of the class is that we’re teaching evidence-based recommendations andinformation. To have them try to find research, read it, and connect to a topic we weredeveloping. … To get them a little excited to know this is part of the research [process], andthey’re contributing, and that it might make a difference”.
In terms of the impact of this approach on instruction, the instructor shared that “students aretaking part in deciding what is important, and making those decisions, and developing thosecritical thinking skills about maybe what … their textbooks have, or don’t have, or that theywish they did have”. The instructor continued by saying, “they have … that student eye, …
and we’re trying to develop them into these critical thinkers who can look at evidence anddetermine ‘Is this worth putting in a textbook, or not?’ … It’s creative, and it makes a lot ofsense”.
UTAUT Constructs with Emerging Themes
The following section reviews the data using the structure of the UTAUT framework in order
to explore the perceptions of the OER team as they developed and applied the resource ininstruction. It also provides data on the perception of OP as applied in the context of this study.
Attitude. This construct refers to an individual’s feelings – both positive and negative –towards the use of a technology (Venkatesh et al., 2003). Three main themes emerged from thedata: pride in localizing the information; a sense of well-being that students were being served;
and sharing of the resource was part of the overall goal.
One of the faculty stated, “Two thirds of our students come from here. So, we really try to be
mindful of that and have the place – our community – reflected in the textbook… with
Hawaiian values and examples… You wouldn’t get that in a traditional publisher’s textbook”.
One instructor, who had recently been a student herself, relayed that “the personal satisfaction
I get is just knowing that I have students who are probably helping their families to pay rent,
or a mortgage, or bills… If I can help to offset that by having this resource available, then you
know, they feel more supported in their education. I think it goes back to equity… and how weare not all privileged”.
Another faculty member reflected on the idea of sharing the OER, “I would be honored to share
the OER … it’s part of the big objective that this be used elsewhere, too”. The technologist
relayed that they “put the most liberal open license on [the OER]. Because we want this content,
and this book, to travel as far as possible”.
Performance expectancy. This construct focuses on the degree to which an individual
perceives that using a system or technology will help in attaining a gain in job performance(Venkatesh et al., 2003). Those interviewed perceived that the application of the OER 1)
afforded benefits to assist students in their learning, 2) provided control over content, and 3)
allowed professional recognition.
Through the interviews, several benefits to students came to light. Data indicated that the OER
benefited students in terms of equity and accessibility when an OER was employed. This wasachieved by both removing the cost barrier for the resource and by opening up forms of access.
One faculty mentioned that, besides being free, one of the main benefits of an OER was “being
able to view it on the web and then also download it in multiple formats”. The OER
technologist also mentioned that “hundreds of students go through this course every semester.

166
168

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

… and they know that they will always have access to the book”. In addition, one faculty sharedthat it had become evident through her experience that “the place-based aspects of [a resource]
help students relate to material, which then typically enhances learning”.
Participants felt that the flexibility of having control over the content in order to revise andupdate the OER was an important strength in using OER. One faculty remarked, “I like theconcept of something that we can have in-house control over. It evolves as we use it. We shouldbe able to improve it and pass it on to the next generation”. The OER technologist recognized
the flexibility using an OER offers: “… it opens up the possibilities of what [faculty] can dowith the book when they know they can change. It’s like being liberated”.
Finally, the team shared a range of professional gains. One faculty didn’t know if work on the
project would be beneficial to promotion because no institutional recognition is given toscholarship when developing OER. However, another faculty felt that the campus promotion
and tenure committee “saw the value” in the work done on the OER because use statistics wereavailable to support instructional efficacy. In addition, the team had received almost 20
requests from different institutions indicating they were interested in adopting the OER, which
impacts networking and professional reputation.
Effort expectancy. This construct refers to the degree of ease associated with the use of thetechnology or system (Venkatesh et al., 2003). It is applied in this research as the ease ofdeveloping the OER and applying it to instruction. Themes that emerged related to thechallenges in developing the OER, and that there were minimal challenges in instruction.
Numerous challenges came to light during the interviews: time needed for resource
development; securing supporting funding; finding and adapting content; and creating
graphics. All team members expressed surprise in regard to the amount of time it took to fully
develop the OER. As one GA mentioned, “The idea of an OER textbook, a free textbook,sounds great, but when you are actually doing a lot of the work, … you realize how much work
it really is!” In addition to recognizing direct costs of developing the OER, one faculty
discussed other indirect costs. The lead faculty estimated the total cost for the GA support for
the entire project as $27,000 and further speculated, “Everything else came through our time –[the time of the faculty and the OER technologist] … so maybe [costing] a total of at least$60,000 to $80,000”.
In regard to implementing the new textbook, all instructors interviewed felt that, rather thanpresenting a challenge, the resource provided a benefit. One instructor mentioned the only real
impact was updating the syllabus.
Social influence. This construct represents the degree to which an individual perceives thatimportant others believe an individual should use the system (Venkatesh et al., 2003). There
was no one outstanding positive social influence identified by the team members. In fact, thelack of social influence was noted more prominently, including from department and college.
Technology self-efficacy. This refers to an individual’s personal belief that he or she possesses
the aptitude and skills to succeed when engaging with a technology (Moghavvemi, 2014). For
this research, this concept revealed the need for skills to create the resource using appropriatetechnology and the technical understanding of open publishing.

167
169

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

“You definitely have to have a pretty solid set of technology skills to be able to create [atextbook]”. This statement by a GA summed up the team perspective about necessary
technology skills. When asked how the skills brought to the project by the OER technologist
contributed to the overall development of the OER, the team, as a whole, felt those skills werecritical. In fact, the technologist shared, “I don’t think that I would have been able to get thisdone in the same amount of time if I wasn’t familiar with open source software and open
content and copyright”.
Facilitating conditions. This is the degree to which an individual perceives that
infrastructures are available for support (Venkatesh et al., 2003). Factors discussed here
include the need for institutional support, and team support that provides expertise in avariety of areas.
One important institutional step in supporting OER development is through a program suchas an OER Grant Project (https://oer.hawaii.edu/projects/). Institutional support also came in
the form of providing the suite of tools with which to create and then host the delivery of the
OER. The OER Technologist mentioned a final form of potential institutional support:“having policies in place for tenure and promotion that value OER production as a scholarly
output”.
The interviews also brought to light how very important a team approach became as the OER
evolved. The OER technologist shared, “if we had a small [consistent] team, a small crew ofpeople that all know how to do [the publishing tasks] … that would make it so much easier”.
One faculty shared how important it was to have an OER technologist as part of the team: “Iwould say [the technologist] was probably one of the most important [members]”.
OER-enabled pedagogy Though only one team member, an instructor, applied OP in
instruction with the newly-developed OER, other team members reflected on the potential
value of exploring OP. Two themes emerged from the interviews: OP has the potential toincrease student involvement in their learning; and the potential is not realized on campus atthat time.
As outlined earlier, one instructor introduced an OP assignment by having students research a
topic of their interest pertinent to the concept of “Health at Every Size”. This instructor shared,“… it makes sense to me … that students are taking part in deciding what is important andmaking those decisions and developing those critical thinking skills about what their textbook
should have or that they wish it did have. … It’s creative and makes a lot of sense”.
In reflecting on OP on campus, the OER technologist indicated, “I think OER-enabledpedagogy showcases what’s possible when the content is open. And I think there’s a lot of
potential in that”. However, he cautioned that “I think it would take some time and some effort
to design pedagogical tools that could be implemented within a course”.
Discussion
This study provides an insight into the perceptions and experiences of one team of university
faculty and staff as they created an OER for an undergraduate course, adopted it in theirinstruction, and explored the use of OP. Findings from this research are important in that they
provide an accounting of the process of OER development and application not seen in the

168
170

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

literature. Themes relating to OP were pedagogical in nature and were, therefore, examinedoutside of the UTAUT framework.
All members of the OER team felt positive about the experience of developing andimplementing the OER. The overall goals were to provide a resource free to students and onethat was place-based and appropriate for local students. Much research has explored thepositive impact on student learning when OER are freely offered (Colvard et al., 2018;
Lieberman, 2018). Other research has indicated the importance of place in culturally relevantscience education (Sutherland & Swayze, 2012). Instructors in this research were particularly
excited about the localization of the content and felt that it positively impacted their teachingexperience with the students, which is consistent with outside research (Ivins, 2011).
Findings for several of the constructs in this research parallel those found in other UTAUT
research on the topic of OER in higher education. “Attitude” was found to be an importantfactor in OER adoption. In their research, Huang and Wu (2013) found that “attitude” was thestrongest predictor of behavioral intention to use OER. In addition, “effort expectancy” wasfound to positively impact the intention to use OER in several studies (Kandiero, 2015; Mtebe& Raisamo, 2014; Padhi, 2018). Finally, findings from this study are consistent with other
OER-related UTAUT findings in regard to the construct of “social influence”. These studies
indicate no social influence on the intention to use or the use of OER (Mtebe & Raisamo, 2014;
Padhi, 2018; Percy & Van Belle, 2012).
There exists an inconsistency with research in the literature and some of the findings in thisresearch. “Performance expectancy” is important in this research, though not in the findings byMtebe and Raisamo (2014). However, in support of this finding, research conducted by Padhi(2018) found that performance expectancy had a positive impact on OER use. The research of
Jhangiani et al. (2016) as well as that of Percy and Van Belle (2012) support this finding. The
most curious and inconsistent finding was regarding the construct of “facilitating conditions”.
The current research revealed its critical importance through both institutional support andteam support. Though the findings of Percy and Van Belle (2012) corroborated this finding in
the current research, those from Mtebe and Raisamo (2014) and Padhi (2018) did not.
Though the importance of technological infrastructure was addressed outside of the scope of
the UTAUT framework in several studies (Mtebe & Raisamo, 2014; Padhi, 2018; Percy & Van
Belle, 2012), technology self-efficacy was not included as a construct in the theoreticalframework within the studies referenced here. The current research found that this construct
was very important as revealed in themes regarding the skills needed to create a resource andin understanding open publishing. These findings are consistent with OER research conducted
outside of the UTAUT framework (Butcher, 2015; Towey et al., 2016).
OER-Enabled Pedagogy
The instructor, who had introduced OP through a research assignment, felt that the students
were developing critical-thinking skills through their involvement. The instructor also felt that
applying this pedagogical approach was creative and would lead to students taking more
responsibility for their learning. In some literature, assignments associated with OP are
identified as non-disposable assignments (NDAs; Wiley, 2013); however, “little organized
literature exists to effectively define, implement, and, accordingly, empirically evaluate the useof NDAs” (Seraphin, 2018, p. 1). Though there is not a plethora of research focusing on OP,
the findings in this research align with the literature, which reveal an excitement and potential
benefit to student learning when OP is applied (Dermody, 2019; Hegarty, 2015; Wiley et al.,

169
171

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 20202017). This research agrees with that found in a study exploring student perceptions of the useof OP as opposed to traditional educational assignments. In that research, OP was perceived as
positive by a majority of students in a multi-institutional study (Hilton III et al., 2019).
The OER technologist pointed out that the development of OER is really the first step in
exploring the freedoms that teaching with OER can afford. In fact, because there is little
direction in the literature on best practices in implementing OP and because the OP assignment
in this research represented an initial first exploration into applying OP, the full potential inthis context has yet to be realized. There is literature that explores the potential of OP for
student learning (DeRosa & Robinson, 2017; Dermody, 2019) but not a great deal of empiricalresearch that has examined the actual potential (Seraphin et al., 2018).
Recommendations
Future research might include more in-depth study into factors that promote or detract from
OER and OP development and use. Future research might also encompass the assessment ofstudent learning when students produce their own learning materials. Some researchers havestressed the importance of localization (Ivins, 2011; Wiley et al., 2014), with recommendations
to localize OER included in the 2012 Paris Declaration on OER (UNESCO, 2012). However,there seems to be little research reporting on this specific concept.
The findings from this research have practical implications as well. They have helped to presenta realistic accounting of the process in developing an OER. Recommendations emerged fromthe data that might be taken into consideration when launching an OER development project.
The recommendations fell into four main categories. Firstly, institutional support is necessary
in order to provide varied incentives such as funding through grants. It is also important toinstitute programs that promote awareness of OER as well as policies that support promotion
and tenure. Secondly, technical support is also necessary. This can take the form of expertisein open resources, accessibility, copyright and licensing, and instructional design. Knowledgeof publishing platforms, design for layout and editing, and image creation are also critical inpromoting the smooth development of OER. Thirdly, OER development can benefit from the
support of a team, who would bring diverse skills to the project. Members would include a
project lead, subject matter experts, and members with necessary technical skills. The final
recommendation is that a time management plan be developed, one that is both flexible andrealistic.
Limitations
One limitation of this research is its generalizability to other contexts (Leung, 2015). It is
recognized that there are differences in the academic culture between various institutions anddisciplines. In addition, the original team for this project was made up of nine individuals, eightof whom agreed to participate in interviews. The small number of participants may affect thegeneralizability of the findings; however, this research was well documented, which can boost
reliability through replication (Leung, 2015).
Conclusion
This study explored the perspectives of faculty and staff engaged with the development andapplication of an OER textbook at an institution of higher education. It applied a qualitativeinterpretive approach through semi-structured interviews with those university professionals

170
172

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

involved with the OER project development and subsequent application in instruction. This
included one instructor’s experience in applying OP.
These findings can help fill a gap in the literature and potentially provide an informed approachto the planning and development of OER. The data from this research helped to outline factors
needed for the support and management of an OER-development project and provided
reflections on the challenges and benefits throughout the process.
The participants in this research have already begun implementing a future edition of the OER.
This will include edits from feedback originating from instructors, students, and otherprofessionals outside of the campus who are using the resource. The upcoming version will
also embed study activities and include a student activity pack and test bank. These features
have shown to be very important for the instructors. The instructor, who experimented with an
OP activity, plans to take the next step in having students assist in developing a new unit for
the OER. Finally, the lead faculty is preparing to develop a new OER for an upper divisionclass for the department. In this research, the process of developing an OER has positively
impacted the participants and has opened up potential projects and creative applications ofOER.
Acknowledgment
The author would like to warmly thank Dr. Christine Sorensen-Irvine, who provided guidance
and feedback on this work, and to those who participated in the study.

171
173

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

References
Algers, A., & Silva-Fletcher, A. (2015). Teachers’ perceived value, motivations for andadoption of open educational resources in animal and food sciences. iJET, 10(2), 35–
45. https://doi.org/10.3991/ijet.v10i2.4427
Anderson, T., Gaines, A., Leachman, C., & Williamson E. P. (2017). Faculty and instructor
perceptions of Open Educational Resources in engineering. The Reference Librarian,58(4), 257–277. https://doi.org/10.1080/02763877.2017.1355768
Belikov, O. M., & Bodily, R. (2016). Incentives and barriers to OER adoption: A qualitative
analysis of faculty perceptions. Open Praxis, 8(3), 235–246.
https://doi.org/10.5944/openpraxis.8.3.308
Brandle, S. M., Katz, S., Hays, A., Beth, A., Cooney, C., Miles, L., DiSanto, J., andMorrison, A. (2019). But what do the students think: Results of the CUNY crosscampus zero-textbook cost student survey. Open Praxis, 11(1), 85–101.
https://doi.org/10.5944/openpraxis.11.1.932.
Butcher, N. (2015). A basic guide to open educational resources (OER). France: UNESCO.
Retrieved from http://unesdoc.unesco.org/images/0021/002158/215804e.pdf
Colvard, N. B., Watson, C. E., & Park, H. (2018). The impact of open educational resources
on various student success metrics. International Journal of Teaching and learning in
Higher Education, 30(2), 262–276.
Corden, A., & Sainsbury, R. (2006). Using verbatim quotations in reporting qualitativesocial research: Researchers’ views. Retrieved fromhttps://www.york.ac.uk/inst/spru/pubs/pdf/verbquotresearch.pdf
Creswell, J. W., Shope, R., Clark, V. L., Green, D. O. (2006). How interpretive qualitative
research extends mixed methods research. Research in the Schools, 13(1), 1–11.
Cronin, C. (2017). Openness and praxis: Exploring the use of open education practices inhigher education. International Review of Research in Open and Distributed
Learning, 18(5). https://doi.org/10.19173/irrodl.v18i5.3096
Cubberley, E. P. (1902). The history of education: Educational practice and progressconsidered as a phase of the development and spread of western civilization.
Retrieved from https://archive.org/details/historyofeducati00cubbuoft/page/n6/mode/2up
DeRosa, R., & Robinson, S. (2017). From OER to open pedagogy: Harnessing the power ofopen. In R. S. Jhangiani, & R. Biswas-Diener (Eds.), Open: The philosophy andpractices that are revolutionizing education and science (pp. 115–124). Ubiquity
Press. https://doi.org/10.5334/bbc.i
Dermody, R. (2019). Open pedagogy for teaching structures. In C. Brause, P. L. Clouston, N.
Darling (Eds.), Building technology educator’s society: Vol 2019, Article 3.
https://doi.org/10.7275/s9xd-h436
Dulle, F. W., & Minishi-Majanja, M. K. (2011). The suitability of the unified theory ofacceptance and use of technology (UTAUT) model in open access adoption studies.
Information Development, 2, 32–45. https:/doi.org/10.1177/0266666910385375
Ehlers, U. D. (2011). Extending the territory: From open educational resources to open
educational practices. Journal of Open, Flexible and Distance, 15(2), 1–10.

172
174

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Hegarty, B. (2015). Attributes of open pedagogy: A model for using Open Educational
Resources. Educational Technology, July-August 2015, 3–13.
Hilton III, J. L., Wiley, D., Chafee, R., Darrow, J., Guilmett, J., Harper, S., Hilton, B. (2019).
Student perceptions of open pedagogy: An exploratory study. Open Praxis, 11(3),275–288. https://doi.org/10.5944/openpraxis.11.3.973
Huang, W. D., & Wu, C. (2013). Motivation system in open learning: A structural equation
modeling on the learners of a traditional Chinese-based open courseware system(MyOOPS). Proceedings of the Association for Educational Communications andTechnology, USA.
Ivins, T. Z. (2011). Localization of open educational resources (OER) in Nepal: Strategies of
Himalayan knowledge-workers. (Unpublished PhD dissertation). Brigham Young University.
Jhangiani, R., Pitt, R., Hendricks, C., Key, J., & Lalonde, C. (2016). BCcampus Research
Report: Exploring faculty use of open educational resources at British Columbia
postsecondary institutions. Retrieved from https://bccampus.ca/wpcontent/uploads/2016/01/BCFacultyUseOfOER_final.pdf
Jung, I., & Hong, S. (2016). Faculty members’ instructional priorities for adopting OER. The
International Review of Research in Open and Distributed Learning, 17(6).
https://doi.org/10.19173/irrodl.v17i6.2803
Kandiero, A. (2015). Educators’ challenges and behavioural intention to adopt Open
Educational Resources: The case of Africa University, Zimbabwe. (Unpublished
Master’s dissertation). University of Cape Town.
Lederman, D. (2018, August 1). Textbook prices’ impact on student behavior. Inside Higher
Education. Retrieved from https://www.insidehighered.com/digitallearning/article/2018/08/01/survey-examines-impact-textbook-prices-student-behavior
Leung, L. (2015). Validity, reliability, and generalizability in qualitative research. Journal of Family
Medicine and Primary Care, 4(3), 324–327. https://doi.org/10.4103/2249-4863.161306
Lieberman, M. (2018). Textbook alternatives take hold at community colleges. Inside Higher
Education. Retrieved from https://www.insidehighered.com/digitallearning/article/2018/05/09/two-year-institutions-report-significant-savings-oer-initiatives
Mallinson, B. J., & Krull, G. E. (2015). An OER online course remixing experience. Open
Praxis 7(3), 263–271. https://doi.org/10.5944/openpraxis.7.3.195
Moghavvemi, S. (2014). Impact of perceived self-efficacy and capability to use IT innovation
on individual use behaviour. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2561739
Mtebe, J., & Raisamo, R. (2014). Challenges and instructors’ intention to adopt and use open
educational resources in higher education in Tanzania. The International Review of
Research in Open and Distributed Learning, 15(1).
https://doi.org/10.19173/irrodl.v15i1.1687
Padhi, N. (2018). Acceptance and usability of OER in Indian higher education: An
investigation using UTAUT model. Open Praxis, 10(1), 55–65.
https://doi.org/10.5944/openpraxis.10.1.623
Percy, T., & Van Belle, J. (2012). Exploring the barriers and enablers to the use of open
educational resources by university academics in Africa. OSS 2012. In I. Hammouda,
B. Lundell, T. Mikkonen, & W. Scacchi, (Eds.), Open source systems: Long-termsustainability, (pp. 112–128). Springer.

173
175

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Reed, J. M. (2018). The history of the textbook: The state of the discipline. Book History, 21,
397–424. https://doi.org/10.1353/bh.2018.0013
Seraphin, S. B., Grizzell, J. A., Kerr-German, A., Perkins, M. A., Grzanka, P. R., & Hardin,
E. E. (2018). A conceptual framework for non-disposable assignments: Inspiring
implementation, innovation, and research. Psychology Learning & Teaching, 18(1),84–97. https:/doi.org/10.1177/1475725718811711.
Sutherland, D., & Swayze, N. (2012). The importance of place in indigenous scienceeducation. Cultural Studies of Science Education, 7(1), 83–92.
https://doi.org/10.1007/s11422-011-9371-1
Towey, D., Ng, R. Y., & Wang, T. (2016). Open educational resources (OERs) andtechnology enhanced learning (TEL) in vocational and professional education andtraining (VPET). 2016 IEEE International Conference on Teaching, Assessment, andLearning for Engineering. https://doi.org/10.1109/TALE.2016.7851808
UNESCO. (2012). 2012 Paris OER Declaration. Retrieved fromhttp://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/Events/English_Paris_OER_Declaration.pdf
U.S. Bureau of Labor Statistics. (2017, August 13). Consumer price index – All urban
consumers: College textbooks. Retrieved fromhttps://www.bls.gov/news.release/archives/cpi_10132017.pdf
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of
information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478.
https://doi.org/10.2307/30036540
Weller, M. (2014). The battle for open: How openness won and why it doesn't feel like
victory. Ubiquity Press. https:// doi.org/10.5334/bam
Wiley, D. (2013, October 21). Iterating toward openness: What is open pedagogy? Retrieved
from https://opencontent.org/blog/archives/2975
Wiley, D., Bliss, T. J., & McEwen, M. (2014). Open educational resources: A review of theliterature. In J. M. Spector, M. D. Merrill, J. Elen & M. J. Bishop (Eds.), Handbook of
research on educational communications and technology (pp. 781–789). Springer
Wiley, D., & Hilton III, J. L. (2018). Defining OER-enabled pedagogy. The International
Review of Research in Open and Distributed Learning, 19(4).
https://doi.org/10.19173/irrodl.v19i4.3601
Wiley, D., Webb, A., Weston, S., & Tonks, D. (2017). A preliminary exploration of therelationships between student-created OER, sustainability, and student success. The
International Review of Research in Open and Distributed Learning, 18(4).
https://doi.org/10.19173/irrodl.v18i4.3022
Yanow, D. & Schwartz-Shea, P. (Eds.) (2006). Interpretation and method: Empiricalresearch methods and the interpretive turn. M.E. Sharpe.
Corresponding author: Beth Tillinghast
Contact email: betht@hawaii.edu

174
176

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

Reviewers Vol. 8, Issue 2
The editorial team would like to thank the following reviewers for their contributions to thepeer review for this issue of the journal. Their dedication and assistance are greatly appreciated.
Philip Ajewole
Hasan Alwadi
Ivy Chia Sook May
Shiao-wei Chu
Melinda Cowart
Heloisa Delgado
Murielle El Hajj Nahas
Mikel Garant
Ana Garcia DiazHee Seon Jang
Leda Kamenopoulou
Michelangelo Magasic
Debbie Meharg
Serpil Meri-Yilan
Elena Mishieva
Leonardo Munalim

Guruprasad Nagraj
Suja Nair
Menyani Nourreddine
Esther Ntuli
Queen Ogbomo
Agnes Papadopoulou
Urarat Parnrod
Anemona Peres
Metee Pigultong
Elena Raevskikh
Shwadhin Sharma
Smriti Singh
Stephanie Wendt
Tony Yeigh
Michelle Yeo

175
177

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020

176
178

IAFOR Journal of Education: Technology in Education

Volume 8 – Issue 2 – 2020176

the iafor

journal of education

ISSN: 2187-0594