the iafor

journal of
education
Volume 7 – Issue 2 – Winter 2019
Editor: Yvonne Masters

ISSN: 2187-0594

iafor
IAFOR Journal of Education
Volume 7 – Issue 2 – Winter 2019

IAFOR Publications

The International Academic ForumIAFOR Journal of Education
EditorYvonne Masters, Australia
Associate Editors
Lynda Leavitt, Lindonwood University, USA
Massoud Moslehpour, Asia University, Taiwan
Raimond Selke, Goethe-Institut Jakarta, Indonesia

Published by The International Academic Forum (IAFOR), Japan
IAFOR Publications, Sakae 1-16-26-201, Naka-ward, Aichi, Japan 460-0008

Executive Editor: Joseph Haldane
Publications Manager: Nick Potts

IAFOR Journal of Education
Volume 7 – Issue 2 – Winter 2019
Published December 1, 2019

IAFOR Publications © Copyright 2019
ISSN: 2187-0594
ije.iafor.org

IAFOR Journal of Education
Volume 7 – Issue 2 – Winter 2019
Edited by Yvonne Masters

Table of Contents
From the Editor
Yvonne Masters

1

Editorial Advice
Yvonne Masters

2

Notes on Contributors

4

Influence of School Heads’ Direct Supervision on Teacher Role
Performance in Public Senior High Schools, Central Region, Ghana
Samuel Yaw Ampofo
George Adino Onyango
Martin Ogola

9

Faculty Target-Based Engagement Assessment Statistical Model for
Enhancing Performance and Education Quality
Mohamed Askar

27

E-learning Self-efficacy of Operating Room Nurses of a Selected
Hospital in Cebu, Philippines
Paul John G. Aventurado

51

eHealth Literacy of High School Students in the Philippines
Mark Kenneth S. Camiling

69

Comparative Study Between Traditional and Peer Assisted Learningof Buisiness Management Students on Accounting Module
Siu Wo Tarloff Im
Pit Ho Patrio Chiu
Michelle Ng

89

Evaluation of Self and Peer Assessments in a Second Year
Engineering Module
David Hassell
Kok Yueh Lee

105

Deision Making Model of Vietnamese Students Studying Higher
Education in England
Minh Hoang
Massoud Moslehpour
Victoria Seitz

131

Academic Staff Induction and Assessment on Implementing
Experiential Learning and Reflective PracticeVera Maria Nistor
Don Amila Sajeevan Samarasinghe

149

Independnce and Interdependence of a Professional
Development School Partnership
Robert F. Sumowski
Joseph M. Peters

169

Reviewers

190

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

From the Editor
Dear Readers,This is the last issue for 2019 and it has been a hectic one. Our indexing with Scopus has led
to a large increase in submissions (140+ for this issue) which has meant increased work foreveryone involved in the journal production: reviewers, editors and publication staff. It has alsomeant the need to restructure the journal for the future, so 2020 looks like being another busy
year. One thing that has occurred in the wake of new indexing is a number of papers submitted
that are out of scope: not about education at all. This has led to development of tighter advice
for 2020 and hopefully clearer guidance for authors.
In this issue, we again demonstrate the multiplicity of fields that mark education. Within thisissue there are papers across a range of disciplines, including health. We are sure that you willfind food for thought and reflection, and the possibility of application in your own fields. Just
as the fields are diverse, our authors also come from several countries, demonstrating theinternational and intercultural nature of our journal.
As we move to a new structure, it is important to remember that a journal issue is not the workof any one person. I would like to thank my Associate Editors, Lynda Leavitt, Massoud
Moslehpour and Raimond Selke, who support me in my role. I also thank the many reviewers
who offer their service to the profession by volunteering to review the many articles submitted
for each issue, providing constructive feedback to authors. These people are the backbone of
the journal. Thanks also go to Nick Potts who takes the files and produces a finished product.
His work behind the scenes is invaluable.
We hope that you enjoy this issue of the IAFOR Journal of Education and consider sharing
your own research and experiences in the journal in the future. The next issue is due to be
published on 1st June 2020.
Yvonne Masters
Editor,
IAFOR Journal of Education

1

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Editorial Advice
Preparing a submission to the IAFOR Journal of Education is more than writing about yourresearch study: it involves paying careful attention to our submission requirements. Different
journals have different requirements in terms of format, structure and referencing style, among
other things. There are also some common expectations between all journals such as the use ofgood academic language and lack of plagiarism. To assist you in reaching the review stage forthis or any other peer-reviewed journal, we provide the following advice which you should
check carefully and ensure that you adhere to.
1. Avoiding Plagiarism
Plagiarism is a practice that is not acceptable in any journal. Avoiding plagiarism is the cardinalrule of academic integrity because plagiarism, whether intentional or unintentional, ispresenting someone else’s work as your own. The IAFOR Journal of Education immediately
rejects any submission with evidence of plagiarism.
There are three common forms of plagiarism, none of which are acceptable:
1. Plagiarism with no referencing. This is copying the words from another source (article,
book, website, etc.) without any form of referencing.
2. Plagiarism with incorrect referencing. This involves using the words from anothersource and only putting the name of the author and/or date as a reference. Whilst not as
grave as the plagiarism just mentioned, it is still not acceptable academic practice.
Direct quoting requires quotation marks and a page number in the reference. This isbest avoided by paraphrasing rather than copying.
3. Self-plagiarism. It is not acceptable academic practice to use material that you havealready had published (which includes in conference proceedings) in a new submission.
You should not use your previously published words and you should not submit about
the same data unless it is used in a completely new way.
2. Meeting the Journal Aims and Scope
Different journals have different aims and scope, and papers submitted should fit the specificjournal. A “scattergun” approach (where you submit anywhere in the hope of being published)
is not sound practice. Like in darts, your article needs to hit the journal’s “bullseye”, it needs
to fit within the journal’s interest area. For example, a submission that is about building bridges,
will not be acceptable in a journal dedicated to education. Ensure that your paper is clearly
about education.
3. Follow the Author Guidelines
Most journals will supply a template to be followed for formatting your paper. Often, there will
also be a list of style requirements on the website (font, word length, title length, page layout,
and referencing style, among other things). There may also be suggestions about the preferred
structure of the paper. For the IAFOR Journal of Education these can all be found here:
https://iafor.org/journal/iafor-journal-of-education/author-guidelines/

2

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

4. Use Academic Language
The IAFOR Journal of Education only accepts papers written in correct and fluent English ata high academic standard. Any use of another language (whether in the paper or the referencelist) requires the inclusion of an English translation.
The style of expression must serve to articulate the complex ideas and concepts being presented,
conveying explicit, coherent, unambiguous meaning to scholarly readers. Moreover,manuscripts must have a formal tone and quality, employing third-person rather than firstperson standpoint (when feasible), placing emphasis on the research and not on unsubstantiatedsubjective impressions.
Contributors whose command of English is not at the level outlined above are responsible forhaving their manuscript corrected by a native-level, English-speaking academic prior to
submitting their paper for publication.
5. Literature Reviews
Any paper should have reference to the corpus of scholarly literature on the topic. A review ofthe literature should:
•
•
•
•
•

Predominantly be about contemporary literature (the last 5 years) unless you are
discussing a seminal piece of work.
Make explicit international connections for relevant ideas.
Analyse published papers in the related field rather than describe them.
Outline the gaps in the literature.
Highlight your contribution to the field.

Referencing
Referencing is the main way to avoid allegations of plagiarism. The IAFOR Journal of
Education uses the APA referencing style for both in-text citations and the reference list. If
you are unsure of the correct use of APA please use the Purdue Online Writing Lab (Purdue
OWL), – https://owl.english.purdue.edu/owl/resource/560/01/ – which has excellent examples
of all forms of APA referencing. Please note APA is used for referencing not for the generalformat of the paper. Your reference list should be alphabetical by author surname and include
DOIs whenever possible.
This short guide to getting published should assist you to move beyond the first editorial review.
Failure to follow the guidelines will result in your paper being immediately rejected.
Good luck in your publishing endeavours,
Dr Yvonne Masters
Editor, IAFOR Journal of Education
https://iafor.org/journal/iafor-journal-of-education/

3

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Notes on Contributors
Article 1: Influence of School Heads’ Direct Supervision on Teacher Role performancein Public Senior High Schools, Central Region, Ghana
Dr Samuel Yaw Ampofo is a full-time lecturer in Educational Administration andManagement at the College of Distance Education, University of Cape Coast, Ghana. He
received his BEd (Foundations) and MPhil (Administration in Higher Education) from the
University of Cape Coast. He obtained his PhD (Educational Administration) from Kenyatta
University, Nairobi- Kenya. He has over nine (9) years teaching experience at the tertiary
education level and has also held various managerial positions. He is currently the Co-ordinatorof Quality Assurance and Enhancement Unit of the College of Distance Education at Universityof Cape Coast. His research interests include issues on financing tertiary education, school
supervision and management, and distance education. He has published in a number of peer
review journals and actively participated in different research endeavours.
Email: sampofo@ucc.edu.gh
Dr George Onyango is a Lecturer in the Department of Educational Management, Policy andCurriculum Studies, Kenyatta University, Kenya. Dr Onyango’s areas of academic interest andresearch include Educational Management, Policy Studies, Principalship, Instructional
Supervision and Educational Research. To date, he has supervised both PhDs and masters
theses and published in a number of internationally referred journals. George holds a BA
(Hons), BEd, MEd, MPhil (Education, Management) from Delhi University and a PhD from
Kenyatta University. He was Chairman, Department of Educational Management Policy andCurriculum Development from 2010-2014. He is currently the Dean, Digital School of Virtualand Open Learning, Kenyatta University as well as the Chairman of the University
Programmes Development Committee. He also sits, as a member, in numerous othercommittees of the University.
Dr Martin Ogola teaches educational leadership and management and is currently the
Chairperson of the Department of Educational Management Policy and Curriculum Studies,
Kenyatta University, Kenya. He holds a PhD and a masters degree in Educational Management
and Policy and a BEd degree from Kenyatta University. Martin is an experienced researcher
with interest in school management and leadership, education policy, and health in schools. He
has been involved as a researcher in several evaluations. He is a member of several professionalorganizations including Kenya Association of Educational Administration and Management(KAEAM), The Society of Educational Research and Evaluation in Kenya Member, and
Education Evidence for Action Network.
Email: martinogola@gmail.com
Article 2: Faculty Target-Based Engagement Assessment Statistical Model for Enhancing
Performance and Education Quality
Dr Mohamed Askar, Professor of Construction Engineering and Management, earned his
PhD in Construction from the Brandenburg University of Technology Cottbus in Germany. He
is currently teaching at the Engineering and Technology Department, Utah State University, in
the USA. He has completed about 30 years in teaching, participating in research programs,
advising MSc and PhD students, and contributing to services to the university, profession andsociety. Dr Askar is an author of several articles, conference proceeding papers, and books. His
research and consultation interests include problems related to the construction industry,4

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019teaching evaluation, and enhancing Engineering education. Dr Askar has more than 25 yearsof experience in managing and leading a group of construction companies in the Middle East,
Asia, Europe and USA.
Email: mohamedaskar@suu.edu
Article 3: E-learning Self-efficacy of Operating Room Nurses of a Selected Hospital inCebu, Philippines
Dr Paul John G. Aventurado holds a PhD in Nursing Science from Cebu Doctors’ University,
Mandaue City, Cebu, Philippines. He is a registered nurse, working as faculty of the university,
specializing in research, medical-surgical nursing and informatics. He is also the researchcoordinator of the CDU College of Nursing, mentor and co-author of CDU Graduate School
research papers, and a peer reviewer of various international journals. He was recently elected
as the president of the Philippine Nursing Research Society Incorporated – CDU cell, and will
spearhead the Knowledge Generation and Publication (KGAP) project of the college. He wasrecently awarded a publication incentive grant from Cebu Doctors’ University. His researchinterests include e-learning, health intervention and theory development.
Email: pjaventurado0521@gmail.com
Article 4: eHealth Literacy of High School Students in the Philippines
Mark Kenneth S. Camiling presently serves as the Lower School Assistant Principal for
Academic Affairs at Miriam College, Philippines. Concurrently, he teaches undergraduate andgraduate classes at the higher education unit of the same institution. He handles courses on
early childhood science and health education, research methods and data analysis techniques
for teachers, and principles of teaching. He obtained both his undergraduate (Bachelor of
Secondary Education-Special Education) and graduate (Master of Arts in Education-Health
Education) degrees at the University of the Philippines Diliman. His Australia Awards
fellowship at Queensland University of Technology in Brisbane, Australia inspired him topursue eHealth literacy as the topic for his master’s thesis. Apart from health education, his
research interests include educational technology and educational psychology He has also
authored numerous health textbooks for elementary and high school students in the Philippines.
Email: mark.camiling@gmail.com
Article 5: Comparative Study Between Traditional and Peer Assisted Learning ofBusiness Management Students on Accounting Module
Mr Tarloff Im is an Education Development Officer at the City University of Hong Kong Office of Education Development and Gateway Education (EDGE). He received his first
degree, BSc in Engineering Physics, from The Hong Kong Polytechnic University, then hegained an MSc in E-Commerce Technologies from The Chinese University of Hong Kong;
MA in Communication and New Media from City University of Hong Kong; and a PgDip in
Education (Professional and Vocational Education) from the Education University of Hong
Kong with Dean’s Honour List. Tarloff has experiences in programme development, quality
assurance of academic programmes, and e-Learning development. His current researchinterests include XR technology for teaching and learning, peer-assisted-learning, and active
learning.
Email: sw.im@cityu.edu.hk

5

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Dr Pit Ho Patrio Chiu is a Senior Education Development Officer at the City University ofHong Kong – Office of Education Development and Gateway Education (EDGE). He received
his BEng (Hons), MEng, and PhD in Electrical Engineering, from McGill University, Canada,
and a PgD Ed from the Hong Kong Institute of Education (now the Education University ofHong Kong). He is named on the Dean’s Honour List at both institutes and is also a Dean’s
Scholar at Queen’s University, Canada. He worked in the R&D industry and served in the
Canadian Space Agency.
Email: patrio.chiu@cityu.edu.hk
Article 6: Evaluation of Self and Peer Assessments in a Second Year Engineering
ModuleDr David Hassell is currently a Senior Teaching Fellow in the Department of Chemical
Engineering at the University of Bath. After obtaining a MEng and PhD in Chemical
Engineering from the University of Sheffield, he undertook research at the University ofCambridge before moving to South East Asia where he taught at three different Universities
within the region prior. He joined his current institution in 2018 and his current researchinterests relate to the application of pedagogical approaches both broadly as well as specificallywithin engineering education.
Email: dgh29@bath.ac.uk
Dr Lee Kok Yueh is a lecturer at the Centre for Communication, Teaching and Learning,
Universiti Teknnologi Brunei, Brunei Darussalam. She received her doctoral degree in English
Language from the University of Birmingham, United Kingdom. Her current research interests
include genre and corpus analyses of spoken and written discourse in areas of communication,
as well as teaching and learning in higher education.
Email: kokyueh.lee@utb.edu.bn
Article 7: Decision Making Model of Vietnamese Students Studying Higher Education in
EnglandMs Minh Hoang, Lecturer in Marketing, earned her MSc in International Marketing from
Sheffield Hallam University in the UK. She currently serves as Senior Lecturer and Associate
Dean of Department of Business Administration at Saigon Institute of Technology, Vietnam.
She has taught for two years and participated in the research, advising students and providing
service to the university profession and society. Her research and consultation interests include
problems related to higher education, customer behavior and satisfaction, and marketing
management.
Email: minhhtb@saigontech.edu.vn
Dr Massoud Moslehpour is an Associate Professor of Foreign Languages, Business, andManagement. He earned his PhD in Industrial Technology from the University of MissouriColumbia. He is currently teaching at the Business Administration Department, Asia
University, in Taiwan. He has 25 years of international teaching and guest lecturing experiencein countries such as the United States, Spain, Iran, Taiwan, Indonesia, Mongolia, Vietnam, andChina. Dr Moslehpour is a distinguished educator and an active researcher. He enjoys advising
MSc and PhD students, contributing to the university, profession, and society. Dr. Moslehpouris the author of several internationally published professional refereed articles, conference
papers, and textbooks. His research and consultation interests include problems related tohigher education, customer satisfaction, and marketing management. Dr. Moslehpour has

6

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019served as the chairman of the Language Center and International Affairs Office in his 25 yearsof working experience.
Email: writetodrm@gmail.com
Dr Victoria Seitz, Professor of Marketing, earned her PhD in Marketing and Merchandisingfrom Oklahoma State University. She currently serves as professor and chair of the Marketing
Department at California State University San Bernardino in the USA. She has taught for over
30 years and participates in research, advising students and providing service to the university
profession and society. Dr Seitz has published five books on Image Marketing and over 100
articles in journals and conference proceedings. Dr. Seitz has received four Fulbright awards
to Romania where she conducted research in advertising and marketing.
Email: vseitz@csusb.edu
Article 8: Academic Staff Induction and Assessment on Implementing Experiential
Learning and Reflective Practice
Ms Vera Maria Nistor is a Lecturer with the English Department at Otago Polytechnic
Auckland International Campus New Zealand, having worked in adult education in different
countries for over 10 years. Holding a Cambridge Delta qualification, she is currently teaching
English for Academic Purposes to international students who are pursuing postgraduate studies
in New Zealand. Vera has also helped facilitate academic staff induction programmes, by
supporting other staff in their use of experiential learning and reflective practice. As a memberof the experiential learning team, she has experience in using the OP Observer digital tool formeasuring the learner-centredness of classes, in order to increase the staff’s quality of teaching.
Her research interests in the field of education include interdisciplinary research on experiential
learning and learner-centred teaching practice and their applicability across different academic
subjects.
Email: Vera.Nistor@op.ac.nz
Dr Don Amila Sajeevan Samarasinghe is a Senior Lecturer in Construction at Otago
Polytechnic Auckland International Campus (OPAIC). He has ten years of experiential
learning and reflective practice-based teaching experience in the New Zealand tertiary
education sector. His qualifications include a PhD in Construction Management (New
Zealand), a Graduate Diploma in Tertiary Education (New Zealand) and a First Class Honours
Bachelor of Civil Engineering degree (Sri Lanka). He is experienced in designing graduate
outcomes, course descriptors, learning outcomes, assessments, marking guides, rubrics, lessonplans and learner capabilities for construction programmes. Dr Samarasinghe has actively
participated in academic staff professional development, supporting other staff and working
collaboratively to enhance the learning experience for students and the teaching experience forstaff. As an active member of the experiential learning team, he used the OP Observer digital
tool to observe and measure the learner-centredness of classes at OPAIC.
Article 9: Independence and Interdependence of a Professional Development School
Partnership
Dr Robert F. Sumowski is an Assistant Professor of Special Education at Georgia College &State University in Milledgeville, Georgia. He holds an Ed D in Educational Leadership fromthe University of Georgia. With a research emphasis on behavior management, Sumowski hasserved on the State Leadership Team of the Positive Behavior Interventions and Supports(PBIS) program of the Georgia Department of Education since 2013 and on the advisory board

7

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

for the Georgia Association of Positive Behavior Support. In 2018-19, he served on the PBIS
Task Force of the Georgia Professional Standards Commission, co-authoring the state
standards and guidance documents for Georgia’s teaching certificate endorsement in PBIS.
From 2015-18, he served as a professor-in-residence assisting Baldwin County (GA) Schools
with PBIS implementation and the reconstitution of the district’s system of behavior
management. Sumowski is a former teacher, behavior specialist, school-based administrator,and district-level director of student safety and management.
Email: robert.sumowski@gcsu.edu
Dr Joseph M. Peters is the Dean of Education at Georgia College & State University inMilledgeville, Georgia. He holds a BA in Elementary Education from Mercyhurst College, an
MEd in Natural Science and Environmental Education from Gannon University, and a PhD in
Science Education from the University of Pittsburgh. His positions include teaching andresearch at the University of West Florida, the University of Florida, Northern Marianas
College, and Chaminade University prior to joining Georgia College & State University. Dr
Peters is the Editor Emeritus of the Journal of Elementary Science Education and author ofnumerous books, chapters, and articles. His research interests include professional
development school partnerships, cultural literacy, and promoting STEM.
Email: joseph.peters@gcsu.edu

8

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Influence of School Heads’ Direct Supervision on Teacher Role Performance in Public
Senior High Schools, Central Region, Ghana
Samuel Yaw Ampofo
University of Cape Coast, Ghana
George Adino Onyango
Kenyatta University, Kenya
Martin Ogola
Kenyatta University, Kenya

9

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
This study assessed the influence of school heads’ direct supervision on teacher role
performance in public senior high schools. The study adopted the embedded mixed methods
design. Slovin’s formula, the proportional allocation method, and simple random and purposive
sampling were used to select a sample of 617 respondents comprising 295 teachers, 222 classprefects, 86 Heads of Department, 13 school heads and 1 Regional Director for the Inspectorate
Division of the Ghana Education Service. Data were collected through questionnaires andinterview guides. Quantitative data were analyzed using frequencies, means and multiple
regression whereas patterns and themes were developed for the analysis of qualitative data.
The study found that school heads allocated very little time for supervision of lesson planningand delivery of teachers. The study established that school heads’ lesson planning supervision(p= 0.043< .05) and lesson delivery supervision (p= .035< .05) had a significant influence onteacher role performance. Therefore, the study recommends the Ghana Education Service todedicate a greater portion of the promotion requirement of the school heads to evidence of
direct supervision of teachers and a reduction in the teaching load of Heads of Department by
the school head to enable them play more instrumental roles in the instructional supervision
process.
Keywords: assessment practices supervision, direct supervision, lesson delivery supervision,
lesson planning supervision, teacher role performance10

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Introduction
Formal education is widely acknowledged to play critical roles in both individual and societal
development. It is considered an investment that accrues both private and social returns andhence, is functional for individual and national progress, irrespective of the level at which it isprovided (Asafo-Adjaye, 2012). For formal education to achieve its goals, key actors such as
school heads and teachers must fully accomplish their roles and responsibilities. Teachers arein the best position to make decisions that directly affect students’ well-being and achievement(Stark, McGhee, & Jimerson, 2017). Therefore, one key concern for success of educational
institutions is to ensure that teachers are well supervised. As Adu, Akinloye and Olaoye (2014)
intimated, supervision (whether internal or external) should be considered a deliberate effort
aimed at enhancing the outcomes of each educational institution. It is a process of involving
teachers in instructional dialogue for the purpose of improving teaching and increasing student
achievement (Sullivan & Glanz, 2013). The term “instructional supervision” refers to the cycleof activities between a supervisor and a teacher targeted at improving classroom performance(Ekyaw, 2014). Undoubtedly, the most important supervision and guidance in the school
setting is that given by the head of the school (Mofareh, 2011). Effective instructional
supervision by the school head is critical to the realization of the outlined objectives of theschool (Mankoe, 2007). These heads carry out instructional supervision through various
instructional supervision practices which includes direct supervision to teachers. The conceptof direct supervision as a form of instructional supervision refers to all the measures by theschool head to facilitate one-on-one feedback with teachers to enhance instruction andprofessional capacity (Glickman, Gordon & Ross-Gordon, 2009). In this study, senior highschool heads’ supervision of lesson planning, lesson delivery and assessment practices wereconsidered.
The key concern of instructional supervision practices by the school head is to improve schools
and students’ achievements by helping teachers to deliver adequately in their role performance(Sergiovanni & Starratt, 2007). Teacher role performance generally includes activities thatteachers professionally perform in the classroom in relation to their areas of specialization(Joshua, Ekanem & Agborbechem 2007). In this study, teacher role performance refers to thedevelopment of good instructional documents, effective lesson delivery, regular assessment ofstudents, regular and punctual school and class attendance, effective use of instructional time,and exhibiting good working relations. In effect, instructional supervision gives teachers
opportunities to collaborate, set goals, understand how their students learn and become better
teachers through improvement in their role performance (Kalule & Bouchamma, 2014;
Sullivan & Glanz, 2013).
Teachers’ schemes of work and lesson plans are the most vital instructional documents that aid
effective instructional delivery. Schemes of work and lesson plans clearly define the structure
and content of a course and map out how resources, class activities and assessment strategies
will be used to ensure attainment of course objectives (Gakuya, 2013). This is confirmed byToo, Kimutai and Kosgei’s (2012) study on the impact of head teachers’ supervision ofteachers on students’ academic performance in the Kenya Certificate of Secondary Education.
The study findings revealed the existence of a positive relationship between head teachers’
inspection of teachers’ instructional documents and performance of students in national
examinations. This implies that head teachers’ inspection of teachers’ instructional documents(schemes of work and lesson plans) is a predictor of students’ performance in national exams.
Thus, in order to ensure effective instructional delivery of teachers, Afolabi and Lato (2008)11

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

recommend that school heads critically examine various items of the lesson plan such asadequacy and relevance, appropriateness and clarity of learner behavioral objectives, andselecting appropriate teaching/ learning resources and evaluation techniques. Researching thestrategies for improving supervisory skills for effective primary education in Nigeria, Enaigbe(2009) describes lesson planning as a reflection of the effort made by the teacher to gather
information for the lesson.
In a study on the relationship between principals’ supervisory strategies and teachers’
instructional performance in primary schools in Delta North Senatorial district, Nigeria,
Osakwe (2010) discovered a significant relationship between the principal’s supervisory
strategies and teachers’ instructional performance in terms of teaching materials and discipline
maintenance. In their study on the impact of selected models of instructional supervision
activities on students’ academic performance in senior high schools in Ondo State, Nigeria,
Alimi and Akinfolarin (2012) established a significant impact of school heads checking ofstudents’ notes, class exercises and visitations, moderation of examination questions andmarking schemes on students’ academic performance in English Language. The study
recommended that school heads must be keen in checking students’ assessment records, suchas notes given by teachers and class exercises, to ensure that teachers are effectively carryingout instructional activities. Thus, the assessment of students in any educational setting is ofparamount importance to the success of such institutions (Ampofo, Bizimana, Mbuthi,
Ndayambaje and Orodtho, 2014). Teachers are expected to regularly assess students with theaim of enhancing students’ performance, whereas school heads oversee the appropriate
execution of teachers’ assigned responsibilities, which includes assessment of students.
Research has shown that school head’s direct supervision of teachers is concerned withimprovement of the conditions that surround learning, pupil growth and effective teacher role
performance in the school system (Alemayehu, 2008). A related study by Glanz, Shulman and
Sullivan (2007) in the United States of America revealed that direct supervision of school heads
in the instructional supervision process focuses on identifying pedagogical challenges
encountered by their teachers in their instructional delivery and providing them with needed
support to overcome them. In the Indian context, Tyagi (2010) emphasized that directsupervision creates a platform for both teachers and school heads to use their collective
expertise in self-appraisal of teachers, to identify gaps in teacher skills, knowledge andcompetencies in order to provide the vital support needed for teachers’ professional
development.
Research studies in Africa have revealed that effective instructional supervision through directsupervision practices of heads contributes to improvement of the education sector. Considering
the case of Kenya, findings of a study by Wanzare (2011) on instructional supervision in public
secondary schools showed that school heads’ direct supervision improves the quality ofteachers and teaching, facilitates students’ academic performance and provides the opportunity
to monitor teachers’ instructional work. Panigrahi’s (2012) study on implementation of
instructional supervision in secondary schools in Ethiopia found that classroom visits enable
head teachers to interact with teachers, determine whether teachers are issuing soundinstruction and provide feedback to help teachers correct highlighted issues. In the case of
Nigeria, a study by Asiyai (2009) showed that regular instructional supervision practices of theschool head through direct supervision of teachers led to improvement in teacher lesson
preparation, regular and punctual class attendance and participation in school community
relations.

12

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

In the Ghanaian context, senior high school heads carry out direct supervision as a key
component of their administrative responsibilities aimed at ensuring teachers perform forrealization of school goals. Direct supervision roles of the senior high school heads include thesupervision of day to day teaching and learning, ensuring the adherence of teachers to theschool time table, facilitating provision of appropriate and adequate instructional delivery byteachers, ensuring punctuality and regularity by both staff and students, and providing direct
assistance on varied issues of concern to teachers (Ghana Education System [G.E.S] as citedin Sekyere, 2014). Recent reports, however, indicate some challenges in teacher role
performance characterized by absenteeism, lateness and poor use of instructional time
(Dickson, 2011). The Metropolitan Annual Performance Reports on Education for Cape Coast
indicates pronounced teacher absenteeism and lateness especially in the first week of reopening
in senior high schools (G.E.S, 2015). Additionally, public senior high schools in the Abura
Asebu Kwamankesse (AAK) district have been performing abysmally in the West African
Senior Secondary Certificate Examination (WASSCE) which is attributed to factors such asteachers’ inadequate pedagogical knowledge and skills, application of inappropriate
approaches of instruction and poor use of instructional time (Osei-Mensah, 2012). Various
studies have continually demonstrated that effective instructional supervision practices byschool heads lead to improvement in teacher role performance, but this is not the case in the
study locale since, teachers are not performing their assigned tasks. This is negatively affecting
senior high schools in terms of non-completion of syllabus, students’ absenteeism and poor
students’ performance.
There is therefore a growing perception among the populace that teachers are not performing
their assigned roles as expected because school heads are not carrying out their directsupervision. However, this remains a perception since there is a dearth of literature on scientificstudies that confirm or deny this assertion. This study therefore sought to explore the extent towhich direct supervision practices (activities) of school heads account for teachers’ role
performance in the study locale. Thus, the current study examines the influence of schoolheads’ direct supervision on teacher role performance in selected public senior high schools inCentral Region, Ghana. Specifically, the study intended finding answers to the following
research questions.
1. To what extent do school heads offer lesson planning supervision to teachers?
2. What level of assistance do school heads offer to teachers through lesson delivery
supervision?
3. To what extent do school heads assist teachers through assessment practices
supervision?
The following research hypothesis guided the study: there is no statistically significant
influence of school heads’ direct supervision to teachers on teacher role performance in public
senior high schools, Central Region, Ghana.
Theoretical Framework
This study was anchored in the General Systems Theory propounded by Ludwig vonBertalanffy. He interpreted “systems” to mean complexes of elements standing in interaction.
The theory investigates the component of a phenomenon, examines the interaction between thecomponents and the relationship that exists between the components and their larger
environment. (Bertalanffy, 1968). In reference to the educational system, the General Systems

13

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Theory recognizes the interdependencies and interrelationship among the parts of the education
system (Jenlink, Reigeluth, Carr, & Nelson, 1996). Thus, any changes in one part of theeducation system is likely to affect other parts of the system. In the school setting, variation ina particular component is likely to result in variations in the interdependence between the othercomponents (students, teachers, school authorities, parents, and many more).
Applying the above specifically to this study, the senior high schools in Ghana in general are
independent systems with various components such as educational authorities, school heads,
students, staff, parents and the community interacting continuously to achieve objectives.
Changes in any of the components are likely to affect the others. Therefore, variations in schoolheads’ supervision of lesson planning supervision, lesson delivery and assessment practices arelikely to cause a change in teacher role performance in senior high schools.
Methodology and Methods
This study employed an embedded mixed methods design (Creswell, 2014; Creswell & Clark,
2011; Onwuegbuzie, 2012) which involved the collection and analysis of both quantitative andqualitative data within a traditional quantitative research design. This design was suitable forthe study because it provided the opportunity for the collection and analysis of both quantitativeand qualitative data (Creswell & Clark, 2011) which involved embedding the two strands ofdata to explain a research problem. It offered the opportunity for the researcher to refine andexplain the quantitative data better (Creswell, 2014).
The target population for the study was 1,638 subjects comprising 1,132 teachers, 406 classprefects, 86 HoDs, 13 school heads and the Regional Director for the Inspectorate Division of
GES. Slovin’s formula (Amin ,2005), a proportional allocation method (Kothari 2013) andsimple random sampling were used to select the teachers, and class prefects, whereas a purposive
sampling technique was employed to select the school heads, Heads of Department (HoDs) and
the Regional Director of the Inspectorate Division of the Ghana Education Service (G.E.S.). The
reliability coefficient of the questionnaires was established using Cronbach’s Alpha formula andwas found to be 0.88 for teachers, 0.76 for class prefects and 0.79 for heads of department. In all,a sample size of 617 respondents comprising 295 teachers, 222 class prefects, 86 HoDs, 13 schoolheads and 1 Regional Director for the Inspectorate Division of GES was used for the study. The
participants for the interviews included all the 13 school heads, 25 teachers (constituting 10% of
the teachers who had already responded to the teachers’ questionnaire) and the Central Regional
Director of the Inspectorate Division of the GES. The main focus of the interview was to ascertain
clarification on some of the emerging issues from the quantitative data analysis. Themes weregenerated from the qualitative data in accordance with the emerging issues and used to support
the quantitative findings. Data were collected through questionnaires and interview guides.
Simple frequencies, percentages, means, standard deviations and multiple regression were usedin analyzing the quantitative data and was presented in tables. Questionnaire items were judged
in accordance with parameters given by Kibuuka, Kiweewa, Nakate & Kizza (2013) andBeinomugisha, Kamya & Said (2014). Thus, calculated mean value of between 4.21 and 5.00 fora particular item meant very high respondents’ agreement with the statement, 3.41-4.20 indicateshigh respondents’ agreement and 2.61-3.40 connotes fair agreement with the statement. On theother hand, a calculated mean value between 1.81 and 2.61 showed low respondents’ agreement/high disagreement with a statement and 1.00-1.80 displayed a very low respondents’ agreement.

14

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Ethical Considerations
The investigator obtained permission from the Central Regional Director of the Ghana
Education Service to establish the authenticity of the researcher’s request for assistance fromheads of the sampled schools. Formal informed consent letters were issued to all respondents
to assure them of confidentiality and anonymity of the information they provided. Respondents
were informed that participation in the study was voluntary and their withdrawal was respected.
Pseudonyms were used for interviewees to conceal their identity. Data gathered wastranscribed verbatim. All sources of information for the study such as writings and researchworks cited were acknowledged through proper referencing.
Results
School Heads’ Lesson Planning Supervision
The first question sought to find out the extent to which school heads offer lesson planningsupervision to teachers in public senior high schools in Central Region, Ghana. Table 1 presents
responses of teachers on the level of lesson planning supervision they received from their
headmasters/headmistresses.
Table 1: Teachers’ responses on direct supervision from school heads through lesson
planning supervision
Sometimes

Seldom

Never

%
23.1

n
32

%
12.7

n
44

%
17.5

n
110

38

15.1

89

35.5

39

15.5

9.2

61

24.3

58

23.1

45

10

4.0

53

21.1

83

33.2

Ensures lesson
objectives are clear
and achievable

29

11.6

66

26.3

77

Offers suggestions
to improve mylesson plans/notes

15

6.0

72

28.7

Helps to selectappropriate
teaching/learning
resources

10

4.0

44

17.5

Lesson PlanningSupervision
Termly inspectionof scheme of work
Discussion ofchallenges on
preparation ofscheme of work
Offers suggestions
to help improve on
the preparation of
my scheme of work
Weekly Vetting oflesson plan/notes

Very Often

Often

n
7

%
2.8

n
58

10

4.0

23

Mean

SD

%
43.8

2.53

1.2

74

29.5

2.51

1.3

17.9

64

25.5

2.65

1.2

56

22.3

49

19.5

2.54

1.3

30.7

29

11.6

50

19.9

2.98

1.3

76

30.3

37

14.7

51

20.3

2.85

1.2

69

27.5

55

21.9

73

29.1

2.45

1.2

Note: Very Often= 5, Often= 4, Sometimes= 3, Seldom=2 and Never=1

15

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

As shown in Table 1, the teachers involved in the study expressed a mostly negative reaction
to the lesson planning supervision they received from their school heads, since most of the
responses had mean values ranging between 2.45 and 2.54. This implies that lesson planningsupervision (ie. termly inspection of schemes of work, discussion of challenges on preparationof schemes of work, weekly vetting of lesson plan/notes) is poorly carried out by school heads
in the study location. It is worth nothing that other aspects of lesson planning supervision (ieensuring clear and achievable lesson objectives and offering suggestions to improve lessonplans) was averagely ensured as depicted in a mean above 2.61. Again, the respondents ranked
the practice of school heads ensuring that the teachers’ lesson objectives are clear andachievable highest (M=2.98) among the lesson planning supervision activities, whereas
helping teachers select appropriate teaching/learning resources was ranked least.
School Heads’ Lesson Delivery Supervision
The second research question was to ascertain the level of assistance school heads offer toteachers through lesson delivery supervision in public senior high schools in Central Region,
Ghana. The responses of the teachers are as captured in Table 2.
Table 2: Teachers’ responses on direct assistance from school heads throughlesson delivery supervision
Very Often
Lesson Delivery
Supervision

Often

Sometimes

Seldom

Never

Mean

SD

n

%

n

%

n

%

n

%

n

%

Ensures that I keep totime for lesson
deliveryClassroom visitationand supervision
Fulfilling outlined
instructional activitiesin lesson plan/notes
Keeping to allocatedinstructional time
Supervising mannerof asking/ distributing
questions andmoderating feedback

54

21.5

91

36.3

39

15.5

26

10.4

41

16.3

3.32

1.4

21

8.4

43

17.1

95

37.8

45

17.9

47

18.7

2.78

1.2

22

8.8

74

29.5

69

27.5

41

16.3

45

17.9

2.94

1.2

28

11.2

70

27.9

70

27.9

33

13.1

51

19.9

2.97

1.3

6

2.4

26

10.4

83

33.1

47

18.7

89

35.3

2.23

1.1

Ensures adequatedelivery of subject
content

24

9.6

47

18.7

71

28.3

46

18.3

63

25.1

2.69

1.3

Note: Very Often= 5, Often= 4, Sometimes= 3, Seldom=2 and Never=1

As reported in Table 2, the majority of the respondents responded positively to the issue ofschool heads’ lesson delivery supervision. The table shows that a mean range of 2.69 to 3.32
for all the items measuring the construct. This implies that most of the respondents agreed withalmost all the items under consideration, except the supervision of the manner of asking/distributing questions and moderating feedback among students which had a mean score of
2.33. The low mean score shows low respondents’ agreement with the statement. Thus, theteachers were generally of the opinion that school heads did not undertake this particular16

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019supervision practice as appropriately as expected. It is worth noting that the item on schoolheads ensuring teachers keep to time for lesson delivery as indicated on the timetable was
ranked highest.
School Heads’ Assessment Practices Supervision
The third research objective was to ascertain the level of supervision teachers receive fromschool heads through assessment practices supervision in public senior high schools in Central
Region, Ghana. The views of the teachers on this issue are as depicted in Table 3.
Table 3: Teachers’ responses on direct assistance from school heads throughassessment practices supervision
Very Often

Often

Sometimes

SeldomNeverAssessment Practices

Mea

SD

n
n

%

n

%

n

%

n

%

n

%

Ensures provision ofopportunity for all
students to participate
fully in lessons
Makes sure that allteachers in the school
receive supervisory
feedbackPraises teachers for
specific teaching
behaviours
Discusses performance of
pupils with teachers

21

8.4

63

25.1

71

28.3

39

15.5

57

22.7

2.81

1.27

27

10.8

67

26.7

67

26.7

43

17.1

47

18.7

2.93

1.27

36

14.3

61

24.3

84

33.5

34

13.5

36

14.3

3.10

1.23

79

31.5

88

35.1

46

18.3

22

8.8

16

6.4

3.76

1.17

Ensures that continuous
assessment records are
kept up-to- date byteachers

96

38.2

101

41.4

23

9.2

13

5.2

15

6.0

4.00

1.11

Note: Very Often= 5, Often= 4, Sometimes= 3, Seldom=2 and Never=1

The figures in Table 3 display a generally positive agreement of the respondents to all the itemson the issue of school heads instructional supervision through the supervision of teachers’
assessment practices. The table shows a mean range between 2.81 and 4.00 for all the items,
implying that the respondents highly agreed with most of the statements. Thus, the respondents
are generally of the view that school heads usually offer supervision to teachers in the form ofsupervising their assessment practices. Notably, the item on ensuring that teachers keep up todate records of students’ continuous assessment was ranked highest.
A null hypothesis was formulated and tested using standard multiple regression analysis to
examine the influence of school heads’ direct supervision of teachers on teacher role
performance in public senior high schools. The results of the Standard Multiple Regression are
as summarized in Table 4.

17

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 4: Standard Multiple Regression analysis of school heads’ direct supervision andteacher role performance
DV: Teacher Role Performance

Constant

Multiple

R Squared

Beta

P-

r

(R2)

(β)

Values

.525*

.043

.204*

.035

.453

.461

PREDICTOR
Lesson Planning Supervision (LPS)
Lesson Delivery Supervision (LDS)

3.817

.354

Assessment Practices Supervision (APS).125

NOTE: *p< .05

Table 4 shows the results of the standard multiple regression of school heads’ direct supervisionof teachers (LPS, LDS and APS) as predictor variable and teacher role performance as adependent variable. Table 4 indicates R2 as .125, implying that 12.5% of variation in teacherrole performance is explained by school heads’ direct supervision to teachers (LPS, LDS and
APS). It is also clear from Table 4 that the p-values for LPS and LDS are lower than thesignificance level, LPS (β=. 525, p= 0.043< .05), LDS (β= .204, p= .035< .05). The p-value
for APS is however higher than the significant level (β= .453, p= .461> .05). This means that
Lesson Planning Supervision (LPS) and Lesson Delivery Supervision (LDS) makesignificantly unique contribution to the prediction of the dependent variable (Teacher Role
Performance). Hence, the study rejected the null hypothesis that there is no statisticallysignificant effect of school heads’ direct supervision to teachers on teacher role performancein public senior high schools in Cape Coast Metropolis and Abura Asebu Kwamankese District,
Central Region, Ghana. This implies that school heads’ direct supervision to teachers is apositive determinant of teacher role performance and thus, improved LPS and LDS would lead
to better teachers’ role performance.
The regression model is as follows:
Predicted TRP = 3.817+0.525(LPS)+0.204(LDS)+ e.
Where TRP= Teacher Role Performance, LPS=Lesson Planning Supervision, LDS= Lesson
Delivery Supervision and e= Error Term.
Discussion
School Heads’ Lesson Planning Supervision
Lesson planning forms a major component of the teaching-learning process. Considering thatthe teachers’ scheme of work constitutes the fundamental basis for any teacher’s professionaldelivery, it is worrying that less attention is paid to its preparation by school heads in the study
locale. As indicated by Gakuya (2013), schemes of work define the structure and content of acourse and clearly outline how resources, class activities and assessment strategies will be used
to ensure that the learning aims and objectives of a course are met. Again, the study by Too,
Kimutai and Kosgei (2012) found a positive relationship between head teachers’ inspection ofteachers’ schemes of work and performance of students in national exams. The school heads’

18

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

poor performance of this responsibility is likely to negatively affect the teacher’s role
performance in terms of preparation of good instructional documents if not checked.
The findings also show that lesson plans are not vetted regularly by the school heads. This
contradicts the expectations from senior high school heads who are admonished to criticallyand consistently examine various items of the lesson plan for effective instructional deliveryof teachers (Mankoe, 2007) as well as the G.E.S. policy on instructional supervision whichauthorizes school heads to regularly vet the weekly lesson plans of all teachers in conformity
to required standards (G.E.S, as cited in Sekyere, 2014). Again, it opposes the position of
Afolabi and Lato (2008) that school heads must check the adequacy and relevance of lesson
notes, appropriateness and clarity of learner behavioural objectives, selecting appropriate
teaching/ learning resources and evaluation techniques as well as offering suggestions where
necessary to improve instructional documents.
On the issue of whether school heads helped their teachers to select appropriate teaching/learning resources for their lessons, the findings show that this is hardly done. This is likely toaffect teachers’ ability to deliver lessons effectively. This finding resonates with the views ofsome of the sampled teachers who were interviewed. One of them lamented that:My school head never makes sure the needed teaching /learning resources areeven provided, let alone help teachers to select the appropriate ones for theirlessons. He is mostly interested in ensuring that we go to class on time. He does
not have time for that. He rather concentrates on other administrative duties.
The findings of the study portray that the school heads scarcely help teachers to choose
appropriate teaching/learning resources for lesson delivery. This finding is at variance with
Osakwe (2010) who posited that there is a significant relationship between the school head’s
supervisory strategies and teachers’ instructional performance in terms of teaching materials.
Osakwe opined that if school heads could help teachers choose appropriate teaching/learning
resources, it could positively influence teaching because that would facilitate the learners’
understanding of abstract concepts (Osakwe, 2010). This implies that there is the likelihood ofstudents in senior high schools not grasping concepts taught in various subjects in the study
locale which may lead to poor students’ performance.
School Heads’ Lesson Delivery Supervision
The results portray high commitment of school heads to the checking of teachers’ punctuality
and effective use of instructional time. These school heads ensured that teachers kept to time
for lesson delivery as indicated on the timetable. This finding is corroborated by the views ofsome of the school heads who were interviewed. One of them affirmed that:
Occasionally, I go round to check if the teachers are in their classrooms
teaching at the time stated on their respective timetables. However, because I
have a lot to do as the head, I have delegated the duty of regular supervision of
instruction to my assistant in charge of academic, but I bear the ultimate
responsibility.
The commitment of the school heads to the practice of ensuring teachers’ punctuality andeffective use of instructional time is in tandem with the view of Enaigbe (2009) who found the
instructional supervision activities of the school head to include seeing to it that teachers

19

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019engage in meaningful instructional activities and keep to allocated instructional time. This
could help to enhance the attainment of instructional objectives, culminate in timely
completion of syllabuses and improve students’ performance.
According to the study findings, school heads visit and supervise teaching in the classroom.
This implies that school heads monitored the instructional delivery of their teachers to offer
assistance when necessary. During the interview majority of the teachers indicated that the visitby the school heads inculcated in them the habit of preparing adequately for their classes. The
monitoring of teachers’ instructional delivery by school heads ties in with the findings ofMalunda, Onen, Musaazi and Oonyu (2016) on instructional supervision and the pedagogicalpractices of secondary school teachers in Uganda which revealed that school heads supervisionof lesson delivery through classroom observations has statistically significant effect on thepedagogical practices of teachers in public secondary schools in Uganda. The findings further
agree with Sergiovanni and Starrat (2007) that informal and formal class visitations byprincipals assist teachers to assess their performance and make conscious efforts to improvesame. The findings of the current study are, however, contrary to the revelation of Malunda,
Onen, Musaazi and Oonyu (2016) that instructional supervision (through portfolio observationand classroom observations) was inadequately carried out and this allowed teachers to employ
ineffective pedagogical practices.
The fact that school heads occasionally checked the adequacy of subject content delivered to
students as found in this study is of great concern. This implies that the school heads spend
very little time on ensuring adequacy of subject content delivered by teachers. This finding iscontrary to that of Sekyere (2014) that school heads must ensure teachers actively involve
students in lessons and adequately deliver subject content.
Lack of knowledge in particular subject specialization or enormity of the duties expected ofschool heads may be blamed for the non-fulfilment of this important responsibility which may
negatively affect role performance of teachers’ lesson delivery.
The revelation that school heads were not bothered about teachers’ active involvement ofstudents in their class is not a good development. The majority of the teachers indicated during
the interview that the major concern of their school heads was teacher regularity andpunctuality to class and not necessarily teachers’ involvement of students in their lessons. Thisfinding is contrary to the findings of Kalule and Bouchamma (2014) that clinical supervisionby school heads have been found to enhance teachers’ instructional delivery through formalclassroom observation by principals in which the principals collect variety of classroom
variables such as teachers’ active engagement of their students in their lessons.
School Heads’ Assessment Practices Supervision
Results on the practice of school heads’ ensuring that continuous assessment records are kept
up-to-date portrays headmasters/headmistresses’ demonstration of a lot of interest andcommitment to this activity. This is likely to reflect in regular assessment of students and makeit easier to identify weaknesses in students’ performance as well as implement corrective
measures on time when the need arises. The finding further agrees with an idea expressed by
the teachers during their interview. In fact, one of them said:
Teachers in my school are expected to complete students’ continuous
assessment records, especially termly reports at least one clear week before

20

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019school reopens for the next term. Teachers who fail to meet set deadlines are
cautioned for the first offence and given queries to answer in subsequent cases.
This can eventually lead to stiffer punishment such as release from the school if
it persists.
The findings also concur with the findings of Alimi and Akinfolarin (2012) who studied theimpact of selected models of instructional supervision activities on students’ academic
performance in Ondo State, Nigeria. The study established a significant impact of schoolheads’ checking of students’ notes, class exercises, moderation of examination questions andmarking schemes on students’ academic performance.
The school heads’ discussion of students’ performance with teachers could have positive
influence on the teachers’ lesson delivery because it will inform teachers of the specific content
areas where students need further assistance. The teachers’ agreement with this item
corroborated the views of some school heads who were interviewed for the study. One of theschool heads concurred by alluding that:
I sit on the academic board at the beginning of the term and agree on the numberof assessments each teacher is supposed to conduct with his/her students. At theend of the term, we have a big forum chaired by the Assistant Head (Academic)
and go through the students’ performance class by class. The form master
responsible reads the performance, we find out where problems are and solvethem.
This view also resonates with the opinion expressed by another school head who reiterated
that:With students’ assessment, I check students’ records and sometimes demand fortheir books. Also, together with my deputies, we meet heads of department todecide on the number of exercises each teacher needs to give to their students
so that I can be checking to ensure they are on course. Sometimes too when we
have hints that a particular teacher is not living up to expectation, I ask thestudents for their exercise books and compare it with the scheme of work.
The views expressed by the teachers and school heads is in agreement with the findings of
Adewale (2014) that school heads’ monitoring /checking of students’ notebooks/exercise
books had a significant effect on academic achievement of students. The finding is also intandem with a World Bank Report (2010) which established that school heads’ monitoring ofteachers’ effective use of instructional time, checking of pupils’ notebooks, giving enoughclasswork, marking assignments, writing and marking corrections enhanced students’
academic performance
From the results, the school heads have shown interest and commitment to ensuring thatteachers receive supervisory feedback from instructional supervisors. This is worthy of notice
due to the crucial role timely feedback plays in the supervision process. It helps to identify
whether their instructional delivery matches the expected standards and ultimately enhance
instructional delivery if effectively carried out (Mapolisa & Tsabalala, 2013).

21

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Opinions expressed by the teachers during the interview session showed that the school heads’
direct supervision to teachers contributed to teachers’ role performance. The teachers indicatedthat regular supervision of their lesson planning and delivery activities as well as supervisionof assessment practices by the school heads instills in teachers, the habit of preparing
adequately before teaching and make them feel their contribution to the school is appreciated.
The results of the multiple regression and views expressed by teachers during the interview
concurs with the findings of Wanzare’s (2011) study which found that direct supervision
practices of school heads in Kenyan public secondary schools, through monitoring teachers’
instructional work, improved quality of teachers and teaching. This finding is further supportedby the study of Panigrahi (2012) on implementation of instructional supervision in Ethiopian
secondary schools. The study found regular classroom visitation by school heads provided
opportunity for headteachers to interact with teachers to know what exactly goes on in theclassroom and ensured teachers performed instructional delivery activities as expected. The
study by Panigrahi (2012) also established that feedback offered by school heads, helped
teachers to rectify anomalies in their instructional delivery process and high standards in theirrole performance. In the Ghanaian context, the findings imply that effective lesson planningand lesson delivery supervision by school heads is likely to enhance teacher role performanceand improve students’ academic performance in public senior high schools.
Limitations
One of the limitations of the study was the geographical location. The Cape Coast metropolis
and the Abura Asebu Kwamankesse district constitute only two out of the twenty-three
metropolitan/municipal and districts that form part of the population. Secondly, the study
sample was small compared to the total population of students in the region. Caution should
therefore be exercised in generalizing the findings to the population. Again, the use of multiple
regression for analysis of the quantitative data fell short of finding out other statistical
implications of the findings such as effect size, confidence interval and determination of
spuriosity.
Recommendations
Based on the findings, the study recommends that:
1. School heads should reduce the teaching load of HoDs to enable them play more
instrumental roles in the instructional supervision component of the school heads’
administrative responsibilities
2. School heads must commit more time and resources to lesson planning and lesson
delivery supervision since they were found to have significant effect on teacher role
performance in senior high schools
3. The Ghana Education Service must dedicate a greater portion of the promotion
requirements of the school heads to evidence of direct supervision of teachers
4. School heads must insist on teachers’ active involvement of students in theirinstructional delivery activities. School heads must organize periodic in-service
training for teachers on various delivery strategies that can be beneficial to theirstudents.

22

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

5. The Ghana Education Service (GES) must ensure the inclusion of group supervisory
approaches for teachers in the official instructional supervision policy for senior highschools, to augment the supervision provided by the school heads.
Conclusion
The primary focus of this study was to examine the influence of school heads direct supervisionon teachers’ role performance in selected public senior high schools in Ghana. There is dearthof studies and limited empirical findings on instructional supervision at the secondary
education level in Ghana. Findings from the study revealed that school heads poorly supervised
lesson preparation by teachers. Thus, instructional documents such as lesson plans/notes andschemes of work were hardly inspected and school heads failed to discuss and give suggestions
to their teachers for improvement. This was mainly due to inadequate time allocated to thisactivity by school heads. However, the study found that school heads moderately supervisedteachers lesson by ensuring teachers’ punctuality, effective use of instructional time anddelivery of lessons in accordance with prepared lesson plans/notes. The study established that
school heads periodically discussed academic performance of students with teachers on regularbasis through departmental and general staff meetings and made sure continuous assessment
records of students were regularly updated. The study therefore concludes that effective schoolheads’ direct supervision will enhance teachers’ role performance in Ghanaian senior highschools. All the necessary resources must be provided for the heads to adequately execute thisimportant administrative function.

23

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
Adewale, O. S. (2014). The positive effect of selected technique of instructional supervision
on students’ academic achievement in secondary schools/high schools. International
Journal of Humanities and Social Science Invention, 3(1), 22–26.
Adu, E.O., Akinloye, G. M., & Olaoye, O.F. (2014) Internal and external school supervision:
Issues, challenges and way forward. International Journal of Educational Sciences,7(2), 269–278, https://doi.org/10.1080/09751122.2014.11890189
Afolabi, F. O. & Loto, A. B. (2008). The headmasters and quality control in primaryeducation through effective intra school supervision in Nigeria. Journal of Teachers
Perspective, 3(2), 4–25.
Alemayehu, G. O. (2008). The current practices and problems of subject-area instructional
supervision in secondary schools of Addis Ababa City Administration (Unpublishedmaster’s thesis). Addis Ababa University, Addis Ababa.
Alimi, O. S. and Akinfolarin, C. A. (2012). Impact of selected modes of instructional
supervision activities on students’ academic performance in senior secondary schools
in Ondo state, Nigeria. Education Research Journal. 2(1), 1–6.
Amin, M., E. (2005). Social Science research: Conception, methodology, and analysis.
Kampala: Makerere University.
Ampofo, S. Y., Bizimana, B., Mbuthi, J., Ndayambaje, I., & Orodho, J.A. (2014).
Performance discrepancies between formative and summative assessments: Focus on
six different higher learning institutions from Ghana, Kenya and Rwanda.
International Journal of Recent Scientific Research, 5(7), 1386–1393.
Arong, F. E., & Ogbadu, M. A. (2010). Major causes of declining quality of education in
Nigeria administrative perspective: A case study of Dekina Local Government Area.
Canadian Social Science, 6(3), 61–76. https://doi.org/10.3968/j.css.1923669720100603.021
Asafo- Adjaye, P. (2012). Private returns on education in Ghana: Estimating the effects ofeducation on employability in Ghana. African Sociological Review, 16(1), 121–139.
Asiyai, R. I. (2009). Effect of clinical supervision of instruction on teacher performance oftasks. Benin Journal of Education Studies, 19(1&2), 238–248.
Beinomugisha, P., Kamya, E, & Said, B. (2014). Ability to deliver and advancement of
women in academic and administrative positions in public universities in Central
Uganda. Educational Research International, 3(2), 36–44.
Bertalanffy, L. V. (1968). General systems theory; Foundations, development andapplications. New York: George Braziller Inc.
Creswell, J.W. (2014). Research design: Qualitative, quantitative and mixed methods
approaches (4th Ed). Thousand Oaks, California: Sage Publications.
Creswell, J. W., & Clark, V. L. P. (2011). Designing and conducting mixed methods research(2nd ed.). Thousand Oaks, CA: Sage.
Dickson, A. (2011). The state of supervision of teaching and learning in public junior highschools in Ghana: A case study of selected junior high schools in Sekyere East24

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

District of Ashanti Region. (Unpublished doctoral thesis), Kwame Nkrumah
University of Science and Technology, Kumasi, Ghana.
Ekyaw, B. A. (2014). The practices and challenges of instructional supervision in Asossa
Zone primary schools. (Unpublished doctoral thesis), Jimma University, Ethiopia.
Enaigbe, P.A. (2009). Strategies for improving supervisory skills for effective primaryeducation in Nigeria. Edo Journal of Counselling. 2(2), 235–244.
https://doi.org/10.4314/ejc.v2i2.60864
Gakuya, N. J. (2013). Influence of headteachers’ instructional supervision practices on
students’ performance in Mathematics in public secondary schools in Nyandarua
South District, Kenya. (Unpublished doctoral thesis), University of Nairobi, Nairobi.
Ghana Education Service. (2015). Metropolitan annual performance reports on education forCape Coast. Ghana: Ministry of Education.
Glanz, J., Shulman, V., & Sullivan, S. (2007, April). Impact of instructional supervision on
student achievement: Can we make a connection? Paper presented at the Annual
Conference of the American Educational Research Association. Chicago, CA.
Glickman, C. D., Gordon, P. S., & Ross-Gordon, J. M. (2009). The basic guide to supervisionand instructional leadership (2nd Ed). Boston: Pearson Education Inc.
Jenlink, P. M., Reigeluth, C. M., Carr, A. A & Nelson, L. M. (1996). An expedition forchange: Facilitating the systemic change process in school districts. Tech Trends,
41(1), 21–30. https://doi.org/10.1007/BF02812078
Joshua, M. T., Ekanem, J. O., & Agborbechem, P. T. (2007). Assessment of professionalroles performance of mathematics teachers in Nigeria. Paper presented at episteme
Conference.
Kalule, L. & Bouchamma, Y. (2014). Teacher supervision practices and characteristics of inschool supervisors in Uganda. Educational Assessment, Evaluation andAccountability, 26, 51–71. https://doi.org/10.1007/s11092-013-9181-y
Kothari, C. R. (2013). Research methodology, methods and techniques, (2nd Revised
Edition). New Delhi: New Age International Publishers, New Delhi.
Kibuuka, M. Kiweewa, E., Nakate, S. & Kizza, S. S. (2013). Small-scale entrepreneurs’
success in Kampala capital authority, Uganda. The Academic Journal of the School ofGraduate Studies, 4(1), 1–16.
Kutsyuruba, B. (2003). Instructional supervision: Perceptions of Canadian and Ukrainian
beginning high-school teachers (Unpublished masters thesis). University of
Saskatchewan: Saskatoon.
Malunda, P., Onen,D, Musaazi, J C. S., & Oonyu, J. (2016). Instructional supervision and thepedagogical practices of secondary school teachers in Uganda. Journal of Education
and Practice, 7(30),117–187.
Mankoe, J. O. (2007). Educational administration and management in Ghana (2nd ed.).
Kumasi: Payless Publication Limited.
Mapolisa, T. & Tshabalala, T. (2013). Instructional supervisory practices of Zimbabwean
school heads. Greener Journal of Education Research, 3(7), 354–362.

25

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Mofareh, A. (2011). School-based instructional supervision in Saudi Arabian public
secondary schools, (Unpublished doctoral thesis), University of York, York.
Onwuegbuzie, A. J. (2012). Introduction: Putting the mixed back into quantitative andqualitative research in educational research and beyond: Moving towards the radical
middle. International Journal of Multiple Research Approaches,6(3), 192–219.
https://doi.org/10.5172/mra.2012.6.3.192
Osakwe, R. (2010). The relationship between principals’ supervisory strategies and teachers’
instructional performance in Delta North Senatorial District, Nigeria. Pakistan
Journal of Social Sciences, 7(6), 437–440.
https://doi.org/10.3923/pjssci.2010.437.440
Osei-Mensah, F. (2012). Factors that influence the performance in general knowledge in Art
of senior high school students in Abura Asebu Kwamankese district in the Central
Region, (Unpublished masters thesis), University of Nairobi, Nairobi.
Panigrahi, M. R. (2012). Implementation of instructional supervision in secondary school: approaches,prospects and problems. Science, Technology and Arts Research Journal, 1(3): 59–67.
Sergiovanni, T. J., and Starratt, R. J. (2007). Supervision: A redefinition (8th ed.). New York:
McGraw-Hill Companies Inc.
Sekyere, E. A. (2014). Teachers’ guide on topical issues for promotion and selection
interviews for teachers and non-teaching staff and general professional update.
Kumasi: Afosek Educational Consult.
Stark, D., McGhee, M. W., & Jimerson, J. B. (2017). Reclaiming instructional supervision:
Using solution-focused strategies to promote teacher development. Journal of
Research on Leadership Education. 12(3), 215–238.
https://doi.org/10.1177/1942775116684895
Sullivan, S., & Glanz, J. (2013). Supervision that improves teaching and learning: Strategies
and techniques (4th ed.). Thousand Oaks, CA: SAGE.
Sullivan, S. & Glanz, J. (2005). Supervision That Improves Teaching (2nd ed.). USA. Corwin Press.
Too, C., Kimutai, C. K., & Kosgei, Z. (2012). The impact of head teachers’ supervision ofteachers on students’ academic performance. Journal of Emerging Trends in
Educational Research and Policy Studies, 3(3), 299–306.
Tyagi, R. S. (2010). School‐based instructional supervision and the effective professional
development of teachers. Compare: A Journal of Comparative and International
Education, 40(1), 111–125. https://doi.org/10.1080/03057920902909485
Wanzare, Z. (2011). Instructional supervision in public secondary schools in Kenya.
Educational Management Administration and Leadership, 40(2), 188–216.
https://doi.org/10.1177/1741143211427977
World Bank. (2010). Supervision and support of primary and secondary education: A policy
note for the Government of Poland. Retrieved fromhttp://documents.worldbank.org/curated/en/425821468093563823/pdf/687220ESW0P1130y0Education0V0Final.pdf.
Corresponding author: Samuel Ampofo
Contact email: sampofo@ucc.edu.gh

26

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Faculty Target-Based Engagement Assessment Statistical Model for Enhancing
Performance and Education Quality
Mohamed Askar
Southern Utah University, USA

27

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
There is a worldwide interest in developing quantitative faculty members’ activity evaluation
models. However, implementing a fair and reliable model is challenging. Without capable andhigh-quality faculty members, no education improvement effort subsequently can succeed.
Based on the gap analysis of the literature, lack of a quantitative faculty member assessment
model might affect teaching and scholarly performance and lead to undesirable effects.
Therefore, most of the existing metrics assessment models do not capture the full range ofactivities that support and transmit knowledge to students.
The main objective of the current research is to develop a practical, comprehensive and flexible
statistical Target-Based Engagement assessment model of faculty members that considers boththe specific faculty needs and the academic unit management concerns. A mathematical
relationship between one or more random and additional non-random variables was used todevelop the model. Descriptive and inferential statistical methods were applied in the data
analysis. The Target-Based Engagement model has seven interconnected aspects and threesubsequent modules. It is a robust statistical framework for automatic faculty assessment.
The results of this model are beneficial for faculty assessment in addition to having wellaligned key performance indicators inside the different levels of the institution. The model
helps in supporting different strategic decision-making of the institution and is considered as along-term improvement method in the academic profession. Creating a vision for future faculty
assessment statistical models will improve the faculty performance and enhance theperformance of all higher education stakeholders.
Keywords: target-based engagement, statistical model, quality of engineering education, selfassessment, faculty member assessment

28

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Faculty evaluation is a complex process that encompasses various interconnected activities andactions, all of which are related to a specific purpose. Furthermore, it plays a vital role ineducation and helping faculty identify areas of strength and weakness in their educational skills.
Without capable and high-quality faculty members, no education improvement effort
subsequently can succeed. Hence, without high-quality evaluation systems, enhancing faculty
members’ performance cannot proceed.
A few studies attempt to evaluate the overall activity of academic staff. Based on the gap analysisof the literature, lack of quantitative faculty member assessment models might affect teachingand scholarly performance and lead to undesirable effects. Therefore, most of the existing
metrics assessment models do not capture the full range of activities that support and transmit
knowledge to students. Significant flaws in both substance and process are present in most ofthem (Elmore, 2008).
Higher Education institutions are facing the challenge of enhancing the quality of academics.
They are usually using student teaching evaluation surveys as a tool to assess the quality ofeducation. These surveys can afford useful information to the management about insights on
their strengths and weaknesses, but evaluating faculty performance requires different parameters
in different criteria to forecast the performance. Using faculty performance assessment criteriasuch as teaching and instruction standards, scholarly activity standards, community service
standards and exceptional development standards will lead to better efficiency and accuracy.
A quantitative assessment of faculty performance aims to identify strengths and weaknesses inorder to provide acceptable professional education enhancement. Accordingly, there are
numerous important challenges in developing a comprehensive and supportive faculty member
evaluation model. The model should:- be based on methodologically sound procedures;- be capable of reflecting differences between academic faculty;- take into account the university/department strategic plans;- help to enhance the performance of faculty members;- be applicable to all faculty members; and- be appropriate as a metric for continuous improvement, promotion and incentives.
Literature Review
There is a worldwide interest in developing quantitative faculty member's activity evaluation
models (Mitchell & Leachman, 2015). However, implementing a fair and reliable model ischallenging. Evaluation of faculty member performance plays an essential role in engineering
education, helping faculty identify areas of strength and weakness in their educational skills.
Lately, numerous state governments of the United States have obviously reduced the investment
in public education (Mitchell & Leachman, 2015), and some states have acknowledged
performance-based distribution to improve state universities regarding faculty development(Dougherty & Natow, 2015; Ellis & Bowden, 2014; Miao, 2012; O’Shaughnessy, 2013;
Umbricht, Fernandez, & Ortagus, 2017). Numerous US public universities exposed toperformance-based financing are currently getting reduced funding from their state legislatures.
Some universities are increasingly having to self-fund some of their programs by raising tuition,
focusing on funding research, and requiring some service fees from their students (Hillman,
Tandberg, & Fryer, 2015). In such cases, the burden of searching for additional funds to support
university programs is carried by the university administrators (chancellors, provosts, deans and29

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019chairs) (Agbetsiafa, 2010). Accordingly, applying diverse techniques for evaluating the qualityof education and faculty members is highly desired.
In reference to the survey data analysis gathered from more than 2,000 American college
administrators, the National Institute for Learning Outcomes Assessment 2009 report (Kuh &Ikenberry, 2009) specifies that faculty engagement is an essential challenge for the assessment
task at higher education institutions. The reason for this engagement is to determine and inspect
empirically the factors that affect faculty members' engagement in learning outcomes-based
assessment. Particularly, it is expected to investigate personal internal factors (e.g., values,
perceptions, attitudes and knowledge) and external factors (e.g., institutional culture, policies
and resources) on their commitment and actual involvement in assessment. As researchers statethat “little is known about faculty and students’ attitudes regarding different aspects ofassessment that have wide-ranging implications for policy and practice in tertiary institutions”(Fletcher, Meyer, Anderson, Johnston, & Rees, 2012, p.119), the conclusions from this faculty
Target Based Engagement (TBE) model will provide a much needed statistical conceptual
framework about faculty's engagement in assessment to develop a quantitative means formeasuring the performance of faculty members. A review of the literature (Banta andAssociates., 2002; Grunwald & Peterson, 2003; Havnes & McDowell, 2008; Maki, 2010; andPalomba & Banta, 2015) discloses investigations of factors that impact faculty members'
engagement in assessment activities, such as (a) time condition; (b) workload; (c) absence ofassessment knowledge and resources; (d) uncertainty about the requirement of assessment; (e)concern of connecting assessment outcomes with faculty evaluation; and (f) fear of assessment
intervening academic self-determination. However, most of the discussions rely on subjective
methods.
Outstanding faculty members are playing the primary role in the assurance and improvement ofteaching and research quality. To support the faculty member assessment system, the university
administration has to provide faculty by methodology and application where used evaluation
models with a scientific background. There are numerous existing different evaluation models
for faculty members. The most known models include the Objective model, the Four levels of
the evaluation model, the CIPP model, Provus’s Discrepancy model, the CIRO model, and theGoal-free evaluation model (Stufflebeam, 2003).
Nowadays, data is at the center of researchers’ work, regardless of whether they deal with
science, industry or education (Chang, Kaufmann, & Kwan, 2014; Provost & Fawcett, 2013).
The availability of such data makes it essential for them to be analyzed and evaluated adequately,
which clarifies the current development of a new field named data science (Hardin et al., 2015;
Emmert-Streib, Moutari, & Dehmer, 2016; Emmert-Streib & Dehmer, 2019; Dehmer & EmmetStreib, 2017). For the analysis of faculty evaluation models, such as regression or classification
methods (Emmert-Streib & Dehmer, 2019), permitting to estimate a forecast faculty model
selection and model assessment are the main concepts for finding the best model for a given data
set. Interestingly, concerning the description of the best model, there are two opposite approacheswith a different fundamental philosophy (Ding, Tarokh, & Yang, 2018; Forster, 2000). One isdefining the best model as predictiveness of a model, and the other as descriptiveness. The latterapproach aims at identifying the accurate model, whose interpretation leads to a deeper
understanding of the generated data and the underlying processes that created the data.
Regardless of the importance of all these model concepts, there are insufficient literature reviews
available on the intermediate level that frame the objectives and approaches of faculty model
selection and model assessment in a limited means. For illustration, innovative literature reviews30

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019presented by Ding et al. (2018), Arlot and Celisse (2010) and Wit, van der Heuvel, and Romeijn(2012) are comprehensive presentations without much detail. Moreover, there are basic outlines
to some models, such as by Aho, Derryberry, and Peterson (2014) and Zucchini (2000). Thesemodels focus only on a minor subsection of the main concepts, making it difficult to distinguishthe broader image of faculty model selection and model assessment.
In addition to the above evaluation models, there are several existing theoretical models for theevaluation of teaching performance presented by Apodaca and Grad (2005), Chen and Hoshower(2003), Mittal and Gera (2013), Seidel and Shavelson (2007), and Scheerens and Bosker (1997).
Apodaca and Grad argued for the theory of teaching effectiveness from a student learning
methodology, in particular, the learning theory. Chen Hoshower applied the expectancy theory,
initially advanced by Vroom, in their exploration of faculty members and student evaluation ofteaching. Mittal and Gera included in their research the teaching effectiveness and charisma
features on student evaluation of teaching effectiveness in higher education. Seidel andShavelson and Scheerens and Bosker reviewed a few faculty effectiveness models that focusedon teaching progression elements and process product models, which positively influenced
student learning aftermaths.
In contrast, the focus of this paper is different in the following ways. First, the general conceptualideas behind the model development, model assessment and their interconnections are presented.
For this, theoretical details are accessible as far as they are helpful for a deeper understanding.
Second, practical methods for the engagement of the faculty and the strategic objectives of thedepartment/university are available as examples of the model inputs. It allows closing the gap
between theoretical understanding and unbiased, practical assessment application. Third, theclarifications aim at an intermediate level of the reader by providing related informationcommonly omitted in advanced texts and forms that should ensure the paper benefits for a wideranging distribution with an overall interest in data science. Finally, some information about thepractical application and validation of the model by using the MatLab R2019a statisticalprogramming are existing. MatLab is used in the statistical assessment because it is a widelyused program, which is freely available and forms the gold standard of the literature on statistics.
Research Paradigm and Hypothesis
This paper does not describe the findings of the research study in detail, but rather, it isrepresenting the design, development, and potential application of the new faculty assessment
model. Consequently, the application gives lesser importance as compared to the model
development due to implementation time limitation that requires collecting data for five years.
Based on the gap analysis of the literature, the study focuses more on model design innovationand concept orientation and does not necessarily require detailed theoretical or experimental
development and analysis.
Hypotheses to be considered when structuring this faculty evaluation model include:- How to design a model indicating the strategic objectives of the department/university anduseful for enhancing faculty performance?
- How to define a clear set of evaluation criteria projecting, in the various areas of academic
activity, stakeholders' values and concerns about academic careers and institutional
policies?
- How to describe, as objectively and explicitly as possible, the performance of each one of
the criteria, considering integrating its quantitative and qualitative dimensions?

31

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Purpose of the Research
The primary purpose of the research is to develop a Target-Based Engagement (TBE) assessment
statistical model of faculty members for enhancing the performance and quality of education. On
the one hand, faculty evaluation has a formative purpose that the results are used to help faculty
development, self-improvement and growth. Moreover, faculty evaluation has a summative
purpose that the results are used to make personnel decisions on faculty promotion andincentives. Secondary objectives are to assess:- Faculty teaching performance with improved specificity of feedback and alignment ofassessments with the students’ assessment survey.
- Faculty self-awareness of their skill level to guide them in targeted teaching skill
acquisition and improvement.
- Institutional teaching competency considering self-evaluations and learner assessment data
are combined to guide programming for faculty development.
Research Methodology
Conceptual Framework of Target-Based Engagement Statistical Model
Figure 1 provides the TBE statistical model conceptual framework summarizing the aspects
involved and the way they interconnect. The main objective is to assure that faculty evaluation
contributes to the improvement of the education quality and student outcomes through enhanced
teaching performance and practice. The framework has seven interconnected aspects.
Persons assessed: Who? Faculty member evaluation is to be analyzed as the vital part of anevaluation and assessment framework of the TBE statistical model, which includes othercomponents such as program/department assessment, teaching/scholarly/service/creativity
assessment, continuous learning evaluation, strengths and weaknesses review, training needs
analysis and strategic plan updating.
Aspects assessed: Which? The model measures the performance of the faculty's contribution
each year that declared in the Faculty Improvement Plan (FIP). The FIP is synchronized with the
Department’s Strategic Plan (DSP). A target variable was constructed based on the engagementof the faculty in selecting the appropriate target activities from the original numeric department’s
Activity Performance Standard Database (APSD) during a specific period. The APSD has four
distinct criteria, as shown in Tables 1-4: teaching and instruction standards, scholarly activity
standards, community service standards, and exceptional development standards. All criterion
items are prepared by faculty in his/her Faculty Improvement Plan and will be compared laterwith the actual achievements. The institution has minimum yearly required points per faculty,
based on its strategic plan. Each criterion item has a different nature of contribution determinedby the institution and based on the degree of importance in the strategic plan. It is classified as
excellent, superior and good.
Furthermore, each criterion is classified into three value categories –superior, excellent, and goodwith different activities worth as follows (Table 5):
1. Each activity in the superior category is worth 3-point values,
2. Each activity in the excellent category is worth 2-point values, and3. Each activity in the good category is worth a 1-point value.

32

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Evaluation technology: How? This aspect refers to the methodology and procedures of a given
approach to faculty evaluation, which is the mix of tools and criteria and standards used in the
TBE faculty evaluation model. Faculty evaluation may be based on a combination of tools suchas self-evaluation, classroom observation, department evaluation, performance indicators,
satisfaction surveys, and external evaluation. It is undertaken concerning criteria and standards
for the teaching profession, such as students learning outcomes, references, and performance. A
statistical analysis model was used for faculty evaluation by applying MatLab.
Capabilities to evaluate and use response: By whom? This aspect concerns the arrangement
to assess, to be evaluated and to use the results of the evaluation. It includes evaluators, such as
faculty, peers, department chair, students and mentors to perform the assessment; and users for
feedback, such as faculty, institution leaders, administrators, policymakers, etc.
Agents involved: With whom? This aspect generally deals with the application aspects of bothfaculty and institution evaluation procedures. Thus, it relates to the participation of a range ofstakeholders such as students, faculty, institution leaders, educational administrators,
policymakers, parents, communities, unions and education professionals.
Aspects gained: What? The main benefits of implementing this model are described in threecriteria groups: faculty assessment, institution evaluation and strategic/decision-making.
Purposes: For what? It includes the objectives of the TBE evaluation model and themechanisms designed to assure that the evaluation results are used in a way such goals arereached. Moreover, the purposes of a faculty evaluation model typically consist of improvement
and accountability. Examples of mechanisms to use evaluation results feedback include
performance feedback, professional development plans, financial and other rewards,
Information/Publication of results and Policy adjustments/development.
Algorithm of Target-Based Engagement Statistical Model
The research set out to develop a TBE assessment model of faculty members. The algorithm of
the TBE statistical analysis module consists of the following modules (Figure 2):
1. Model Inputs: Faculty Member Activity Plan
1.1. Category Level:
Equation (1) represents an example of the planned assessment points of a superior
category (teaching standards criterion) that worth 3-points per activity.
:

𝑇𝑒𝑎𝑐ℎ𝑖𝑛𝑔 𝑆𝑢𝑝𝑒𝑟𝑖𝑜𝑟 𝐶𝑎𝑡𝑒𝑔𝑜𝑟𝑦 𝑃𝑙𝑎𝑛𝑛𝑒𝑑 𝑃𝑜𝑖𝑛𝑡𝑠 = 7 3. 𝑇𝑆𝑖

(1)

;<=

Where;
TSi = Teaching superior category planned activities (TS1, TS2, …….., TSn),
3 = Worth points for each activity included in the superior category, andn = Number of selected activities
Equation (2) represents an example of the planned assessment points of an excellent
category (scholarly activity standards criterion) that worth 2-points.
:

𝑆𝑐ℎ𝑜𝑙𝑎𝑟𝑙𝑦 𝐸𝑥𝑐𝑒𝑙𝑙𝑒𝑛𝑡 𝐶𝑎𝑡𝑒𝑔𝑜𝑟𝑦 𝑃𝑙𝑎𝑛𝑛𝑒𝑑 𝑃𝑜𝑖𝑛𝑡𝑠 = 7 2. 𝑆𝐸𝑗
E<=Where;
SEj = Scholarly excellent category planned activities (SE1, SE2, …….., SEn),

33

(2)

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

2 = Worth points for each activity included in the excellent category, andn = Number of selected activities

Figure 1: The Conceptual Framework for TBE Faculty Evaluation Model

34

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 1: Faculty teaching activities plan
Engineering & Technology Dept.

Faculty Activities Plan

Faculty Name: ABC

Final Assessment
Adjusted Progress,
Activities Points Activities Points Quality Points%
Proposed Plan

Plan Activities

Actual Achievement

1. Teaching (Superior Performance = 18 Points, Excellent performance = 14 Points, Good Performance = 10, Unacceptable Performance = 4)
1.A. Superior Value (3 points) "Direct Evaluation"
1.A.1
1.A.2
1.A.3
1.A.4
1.A.5
1.A.6
1.A.7
1.A.8
1.A.9
1.A.10

15.00

Review teaching evaluations based on department standard and implement a
research-based change (required each year)
Attend a half-day CETL workshop and implement a research-based teachingchangeAttend a CETL teaching circle that meets multiple days and implement a
research-based change
Complete a significant formative assessment of student learning andimplement a research-based change
New preparation for a three-hour course
Teaching Effectiveness and ability
Course visiting/observations of classroom presentation, teaching style andinstructional methods and tools
Syllabi, course objectives, course scope, depth and sequence of course
material
Teaching and examination methodology, material and using Canvas
Student Learning Outcomes

1.B.2
1.B.3
1.B.4
1.B.5
1.B.6
1.B.7
1.B.8
1.B.9
1.B.10

3.00

1

3.00

75%

2.25

75%

1

3.00

1

3.00

90%

2.70

90%

1

3.00

1

3.00

85%

2.55

85%

1

3.00

-

0%

1

3.00
-

2.70
-

90%
0%

1.C.4
1.C.5
1.C.6
1.C.7
1.C.8
1.C.9
1.C.10

1

3.00
-

90%

-

-

-

0%

-

-

-

0%

-

-

-

0%
0%

4.00

Professional Knowledge - understanding curriculum, subject content, andstudent needs
Instructional Planning - using SUU curricula and standards, effective
strategies, and resources, to address student needs
Instructional Strategies - engaging students in active learning to facilitate thestudents’ acquisition of vital knowledge and skills
Differentiated Instruction - challenging student’s learning by providing
appropriate content which addresses individual learning differences
Assessment Strategies - choosing a variety of diagnostic, formative, andsummative assessment strategies that are valid and appropriate for thecontent
Assessment Uses - measuring student progress, to inform instructional
content and delivery methods
Positive Learning Environment - providing a well-managed, safe, andorderly environment that is conducive to learning
Academically Challenging Environment - creating a student-centered,academic environment in which teaching and learning occur at high levels
and students are self-directed learners
Professionalism - exhibiting a commitment to professional ethics and thedepartment’s mission and participating in professional growth opportunities
to support student learning
Other report/activity deemed (by the Department Chair) to be of similarcaliber

3.40

1

2.00

1

1

1.C. Good Value (1 point) "Any other Pieces of evidence"
1.C.1
1.C.2
1.C.3

68%

1

1.B. Excellent Value (2 points) "Reports" at least 1 / semester
1.B.1

10.20

2.00

2.00

90%

1

2.00

80%

Sub-Total (Teaching)

1

1.80

90%

-

0%

1.60

80%

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

1.00

1.00

1

1.00

100%

1

1.00

2.00

100%

1.00

0%
0%
100%

100%

1.00

0%
100%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

21.00

35

0%

-

1

-

-

2.00

Attend a half-day CETL workshop without implementing a change
Mentor a senior design group
Review one teaching-related conference or journal paper
Assess the impact of a continuous improvement item using an appropriate
methodology
Evidence of creation of an organized learning environment (syllabi, course
outlines, learning objectives, study guides, etc.)
Evidence of creative, thoughtful and thorough methods and materials(electronic media, unique field experiences, lab experiences, classroom
activities, projects, etc.)
Evidence of seeking and receiving feedback from students and others aboutteaching performance (student scores and comments, peer evaluations ofclassroom and/or materials)
Evidence of thoughtful reflection about the feedback (analysis ofquantitative and qualitative data, summary, overviews, etc.)
Evidence of adjustments made (comments about what worked and didn't
work and thoughts of why)
Other evidence/activity deemed (by the Department Chair) to be of similarcaliber

85%

0%
15.60

74%

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 2: Faculty scholarly activities plan
Engineering & Technology Dept.

Faculty Activities Plan
Proposed Plan

Plan Activities

Faculty Name: ABC
Actual Achievement

Activities Points Activities Points Quality

Final Assessment
Adjusted Progress,
Points
%2. Scholarly (Superior Performance = 12 Points, Excellent Performance = 9 Points, Good Performance = 5, Unacceptable Performance = 2)
2.A. Superior Value (3 points)
2.A.1
2.A.2
2.A.3
2.A.4
2.A.5
2.A.6
2.A.7
2.A.8
2.A.9
2.A.10

12.00

2.B.5

-

-

-

0%

1

3.00

-

-

0%

1

3.00

Present an online short course in connection with a conference byinvitation or request
Conducting a workshop or formal training for SUU faculties and staff
Participation in projects that result in a Funded External Grant over $40,000(Principal or Co-Principal Writer) OR Director of a multi-year project
involving over $100,000 in funding
Implementation results of faculty/student scholarly projects or activities
Formal review of a college/department program by request
Pedagogical research or work in a successful ABET accreditation (Principal
Writer)Develop a course that results in distribution (presenting it at a conference,for example)

1

Poster or oral presentation at scholarly/professional venue
Work with students who present a poster/ paper at professional meetings
Publication of a book chapter or section
On-line workshop or presentation
Presentation at a national or international professional event or conference

Contributor for national accreditation or accreditation review

2.B.10

Reviewing, creating, investigating, or applying software applications in new
ways

1

1

2.C. Good Value (1 point)

2.C.2
2.C.3
2.C.4
2.C.5
2.C.6
2.C.7
2.C.8
2.C.9
2.C.10

1

3.00

2.55

85%

-

-

-

0%

-

-

-

0%

3.00

-

-

0%
0%

-

-

-

0%

-

-

-

0%

1

3.00

100%

85%

4.00

2.B.9

2.C.1

100%

3.00

2.B.8

2.B.7

3.00

1

Participation in projects that result in a Funded External Grant over
$20,000.00-$39,000 (Principal or Co-Principal Writer) OR Director of themulti-year project involving $40,000-$99,999 in funding
Engaging in scholarly activity that gives rise to the improvement ordevelopment of curriculum
Referee an article in an online venue or journal OR review a text or lab
manual by request of the publisher

2.B.6

46%

Formal collaborative undergraduate research (faculty-student) that will
result in dissemination
Publication of a book or commercial lab manual

2.B. Excellent Value (2 points)
2.B.1
2.B.2
2.B.3
2.B.4

5.55

Publication in peer-reviewed venues (conference proceedings or journals)

1.80

-

-

-

0%
0%
0%
0%

-

-

-

0%

-

-

-

0%

2.00

-

-

0%

-

-

-

0%

-

-

-

0%

1.80

90%

0.75

75%

2.00

1

2.00

90%

1.00

Published article in a non-refereed journal or other print or electronic
mediumWork with students who present a poster/paper at a non-refereed journal
or other print or electronic medium
Book review for the publishing company
On-line related-venue
Conducting a formal training or workshop OR serving as a guest lecturer in acolleague’s class
Professional consultation report which is submitted in writing to a clientthat has local impact OR Co-PI of multi-year funded grant ($40,000-$99,999)
Be nominated for a scholarly award
Review an article or five abstracts in a journal, conference or online venue
Develop pedagogy that results in dissemination (presenting at a conference,
CARAT, etc.)
Developing assessment criteria, methods or materials
Sub-Total (Teaching)

-

1.00
-

-

1

1.00
-

75%

0%

-

0%

0.75
-

75%
0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

17.00

36

-

1

45%

8.10

0%
48%

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 3: Faculty service activities plan
Engineering & Technology Dept.

Faculty Activities Plan

Faculty Name: ABC

Final Assessment
Adjusted Progress,
Activities Points Activities Points Quality Points%
Proposed Plan

Plan Activities

Actual Achievement

3. Service (Superior Performance = 10 Points, Excellent Performance = 6 Points, Good Performance = 3, Unacceptable Performance = 1)
3.A. Superior Value (3 points)

6.00

5.55

93%

3.A.1

Chairing any Departmental, College, or University committee

-

-

-

3.A.2

Serving on any hiring committee
Working as Engineering Club advisor [due to the planning timecommitment]
Working in an official position for a regional or national professionalsociety, or a state entity (such as USHE or USOE)
Accompanying students to regional, national or international conferenceswhere students’ original or collaborative work is presented
Serving on the Faculty Senate
Working on an LRT Committee (at any level)

-

-

-

0%

2.55

85%

3.A.3
3.A.4
3.A.5
3.A.6
3.A.7
3.A.8

1

1

Other achievements /activities deemed by the Department Chair to be ofsimilar caliber

3.B.5
3.B.6
3.B.7
3.B.8

1

3.C.2
3.C.3
3.C.4
3.C.5
3.C.6
3.C.7
3.C.8

85%

-

-

-

0%

-

-

-

0%

3.00

3.00

3.00

1

100%

-

-

2.00

1

2.00

100%

0%
100%
0%

2.00

100%

2.00

0%
0%
0%
100%

-

-

-

0%

Working as a course coordinator for a Departmental course

-

-

-

0%

Service in the SUU community (including other committees)
Service as a supervisor of an organization or student club
Other achievements/activities deemed by the Department Chair to be ofsimilar caliber

-

-

-

0%
0%

-

-

-

0%

3.C. Good Value (1 point)
3.C.1

3.00

2.00

Serving on the Provost’s Retention Committee
Working on the College Recruitment and Retention Committee
Working on the Departmental Recruitment and Retention Committee
Serving on the Department Curriculum Committee
Serving on the Undergraduate Research Committee

3.B.9

1

-

3.B. Excellent Value (2 points)
3.B.1
3.B.2
3.B.3
3.B.4

3.00

0%

2.00

Membership in a professional organization (ASEE, AIAA, ASCE, ASME, ASM,
ASTM, IEEE, etc.)
Be designated for a service award or other professional recognition
Work as a supervisor of a group preparing for a non-technical competition
Engineering-related service in the non-SUU community
Serving as a Science Fair judge

1Serving on the University Finance Committee

1.75

-

-

-

0%

1.00
-

1.00
-

0.75
-

0%
75%
0%
0%

-

0%

1

-

Serving on the University Honors Committee

1

Other achievements/activities deemed by the Department Chair to be ofsimilar caliber
Sub-Total (Service)

1.00
10.00

37

88%

75%

1

1.00
-

100%

1.00

100%

-

0%

9.30

93%

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 4: Faculty exceptional development activities plan
Engineering & Technology Dept.

Faculty Activities Plan

Faculty Name: ABC

Final Assessment
Adjusted Progress,
Activities Points Activities Points Quality Points%
Proposed Plan

Plan Activities

Actual Achievement

4. Exceptional Development (Superior Performance = 10 Points, Excellent Performance = 6 Points, Good Performance = 3, Unacceptable
Performance = 1)
4.A. Superior Value (3 points)
4.A.1
4.A.2
4.A.3
4.A.4
4.A.5
4.A.6
4.A.7
4.A.8
4.A.9
4.A.10

6.00

External or university awards, honors or other recognition for intellectual
contributions
Processes used for and the development of the existing program; such as
CanvasLeading the process of setting, monitoring andachieving specific and challenging goals or strategies that reflect highexpectations
Leading an implemented departmental program/workshop shows the visionof continuous improvement
Building any data-tracking systems
Leading a project that represents department improvement; such as(Strategic Plan)
Service to professional organizations or publishing applicable papers,
studies or projects to support the profession
Leading a leadership team
Leading a staff development program; such as the leader of an enhancing
mechanism
Other activities/achievements deemed by the Department Chair to be ofsimilar caliber

1

4.B.2
4.B.3
4.B.4
4.B.5
4.B.6
4.B.7
4.B.8

3.00

1

3.00

1

4.B. Excellent Value (2 points)
4.B.1

5.25

3.00

100%

1

3.00

75%

Sharing in analysis work of measuring Value-Added to students at the endof any semester
Participating in any lessons learning workshops

1

1

0%

3.00

100%

-

0%

2.25

75%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

-

-

-

0%

4.00

Participating in an implemented departmental program/workshop shows
the vision of continuous improvement
Participating in a project that represents department improvement; such as(Strategic Plan)
Participating in a leadership teamParticipating in a staff development program; such as a member of anenhancing mechanism
Sharing in analysis and revision of curriculum, instruction, assessments andallocation of resources to ensure alignment of courses with SUU standards
Build group conversations– topics and agendas

88%

1.50

38%

-

-

-

0%

-

-

-

0%

-

-

-

0%

1.50

75%

2.00

1

2.00

75%

-

-

-

0%

-

-

-

0%

2.00

-

-

0%

-

-

-

0%

4.B.9

Sharing in activities related to a shared vision of continuous improvement

-

-

-

0%

4.B.10

Other activities/achievements deemed by the Department Chair to be ofsimilar caliber

-

-

-

0%

4.C. Good Value (1 point)
4.C.1
4.C.2
4.C.3
4.C.4
4.C.5
4.C.6
4.C.7

2.00

Professional development for self and staff – notes, agendas, minutes
Any documentation processes for the department activities to be used laterin any continuous development work
Scheduling site visits from education associations or industry (at least
2/year)
Allocate resources, including technology, to assist student and staff learning
Create a collaborative learning culture

1

Staff meeting observations for problem-solving
Following-up department meeting minutes

0.75

38%

-

-

-

0%

-

-

-

0%

1.00

0.75

75%

-

1

1.00
-

75%

-

0%
0%
0%
0%

4.C.8

Booster club information

1.00

-

-

0%

4.C.9

Following-up programs that allow alumni to return and give back

-

-

-

0%

4.C.10

Other activities/achievements deemed by the Department Chair to be ofsimilar caliber

-

-

-

0%

1

Sub-Total (Creativity)

12.00

7.50

63%

Total Assessment, based on Faculty Plan

60.00

40.50

68%

Total Assessment, based on Required

50.00

40.50

81%

38

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 5: Relative weights of the department’s activity performance standard database module

Assessment Category
Teaching
Scholarly
Service
Exceptional Development
Total

Model Relative Weights
Superior
Excellent
Score
%age
Score
%age
18
36%
14
39%
12
24%
9
25%
10
20%
6
17%
10
20%
7
19%
50
100%
36
100%

Good
Score
10
5
3
3
21

Figure 2: Engagement assessment statistical model of faculty members

39

%age
48%
24%
14%
14%
100%

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Equation (3) represents an example of the planned assessment points of a good
category (service activity standards criterion) that worth 1-point per activity.
:

𝑆𝑒𝑟𝑣𝑖𝑐𝑒 𝐺𝑜𝑜𝑑 𝐶𝑎𝑡𝑒𝑔𝑜𝑟𝑦 𝑃𝑙𝑎𝑛𝑛𝑒𝑑 𝑃𝑜𝑖𝑛𝑡𝑠 = 7 1. 𝑉𝐺𝑘

(3)

J<=

Where;
VGk= Service good category planned activities (VG1, VG2, …….., VGn),
1 = Worth points for each activity included in the good category, andn = Number of selected activities
1.2. Criteria Level:
Equation (4) represents an example of the planned assessment points of a criteria
level (case of teaching activity standards criterion).
:::

𝑇𝑒𝑎𝑐ℎ𝑖𝑛𝑔 𝑃𝑙𝑎𝑛𝑛𝑒𝑑 𝑃𝑜𝑖𝑛𝑡𝑠 = 7 3. 𝑇𝑆𝑖 + 7 2. 𝑇𝐸𝑗 + 7 1. 𝑇𝐺𝑘
;<=

E<=

(4)

J<=

Where;
TSi = Teaching superior category planned activities (TS1, TS2, …….., TSn),
TEj = Teaching excellent category Planned Activities (TE1, TE2, …….., TEn),
TGk= Teaching good category planned activities (TG1, TG2, …….., TGn),
3 = Worth points for each activity included in the superior category,
2 = Worth points for each activity included in the excellent category,
1 = Worth points for each activity included in the good category, andn = Number of selected activities
1.3. Faculty Improvement Plan Level:
Equation (5) represents the total planned assessment points of the faculty member
improvement plan.
:::

𝑇𝑜𝑡𝑎𝑙 𝑃𝑙𝑎𝑛𝑛𝑒𝑑 𝑃𝑜𝑖𝑛𝑡𝑠 = 7 3. 𝑇𝑆𝑖 + 7 2. 𝑇𝐸𝑗 + 7 1. 𝑇𝐺𝑘 +
;<=
:

E<=
:

J<=
:

7 3. 𝑆𝑆𝑖 + 7 2. 𝑆𝐸𝑗 + 7 1. 𝑆𝐺𝑘 +
;<=
:

E<=
:

J<=
:

7 3. 𝑉𝑆𝑖 + 7 2. 𝑉𝐸𝑗 + 7 1. 𝑉𝐺𝑘 +
;<=
:

E<=
:

J<=
:

7 3. 𝐸𝑆𝑖 + 7 2. 𝐸𝐸𝑗 + 7 1. 𝐸𝐺𝑘
;<=

E<=

(5)

J<=

2. Model Processing: Faculty Member Actual Measurements
The statistical model was used to represent, frequently in the significantly idealized form,
the data-generating process. The model was specified as a mathematical relationshipbetween one or more random variables and additional non-random variables. Two main
statistical methods were used in the data analysis: descriptive statistics, whichsummarized data from a sample using indices such as the mean or standard deviation,and inferential statistics, which concluded from the data that were subject to randomvariation. The actual points of the yearly FIP were compared with the planned ones tocreate a score by using the above formulas. The value categories were automatically40

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

awarded and calculated for all different levels of the university, department, and faculty
by using MatLab R2019a. The model calculates the total faculty actual performance inthree levels:
2.1. Category Level.
2.2. Criteria Level.
2.3. Faculty Improvement Plan Level:
Equation (6) represents the total actual assessment points of the faculty member.
:::

𝑇𝑜𝑡𝑎𝑙 𝐴𝑐𝑡𝑢𝑎𝑙 𝑃𝑜𝑖𝑛𝑡𝑠 = 7 3. 𝐴𝑇𝑆𝑖 + 7 2. 𝐴𝑇𝐸𝑗 + 7 1. 𝐴𝑇𝐺𝑘 +
;<=
:

E<=
:

J<=
:

7 3. 𝐴𝑆𝑆𝑖 + 7 2. 𝐴𝑆𝐸𝑗 + 7 1. 𝐴𝑆𝐺𝑘 +
;<=
:

E<=
:

J<=
:

7 3. 𝐴𝑉𝑆𝑖 + 7 2. 𝐴𝑉𝐸𝑗 + 7 1. 𝐴𝑉𝐺𝑘 +
;<=
:

E<=
:

J<=
:

7 3. 𝐴𝐸𝑆𝑖 + 7 2. 𝐴𝐸𝐸𝑗 + 7 1. 𝐴𝐸𝐺𝑘
;<=

E<=

(6)

J<=

3. Model Outputs: Statistical Analysis of Faculty Assessment
The TBE model outputs were figured based on the statistical experiments that can besummarized in the following:
1. Direct outputs: Dashboard of faculty performance indicators (Figures 3-14). They
are those outputs that calculated and developed directly in the model.
2. Indirect Outputs: Dashboard of the institution's performance and strategic
decision-making indicators. They are outputs that need more processing andcalculation to figure out the indicators.
TBE Analysis and Implementation
The TBE model is a robust statistical framework for automatic faculty assessment. It is a
simplified, mathematically-formalized way and optionally to make predictions from thisapproximation to analyze the performance measurements and help in the institutional
development decision-making. It measures the performance of the faculty's contribution in agiven year. The data were collected from the faculty activity development plan in theEngineering and Technology Department at Southern Utah University in the year of 2018-2019.
For ethical consideration and confidentiality, all collected data were without compromising theidentities of their respondents.
Table (1) shows that the total proposed faculty teaching activities plan was 21.00 points, while
the final adjusted points were 15.60, with achievement progress of 74%. Table (2) shows thatthe total proposed faculty scholarly activities plan was 17.00 points, while the final adjusted
points were 8.10, with achievement progress of 48%. Table (3) shows that the total proposed
faculty service activities plan was 10.00 points, while the final adjusted points were 9.30, withachievement progress of 93%. Table (4) shows that the total proposed faculty exceptional
development activities plan was 12.00 points, while the final adjusted points were 7.50, withachievement progress of 63%. The total assessment was 60 points, based on the proposed plan,
while the total final adjusted points were 40.50, with total achievement progress of 68%. The

41

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

total assessment was 50 points, based on the required achievements from the institution, while
the total final adjusted points were 40.50, with total achievement progress of 81%.
Based on the institution's strategic plan, table (5) shows that the model teaching score designedfor ‘superior’ as 18, ‘excellent’ as 14, and ‘good’ as 10. The model scholarly score designed for‘superior’ as 12, ‘excellent’ as 9, and ‘good’ as 5. The model service score designed for ‘superior’as 10, ‘excellent’ as 6, and ‘good’ is 3. The model exceptional development score designed for‘superior’ as 10, ‘excellent’ as 7, and ‘good’ as 3.
The engagement of the faculty in achieving the activity performance of the institution was givenemphasis on all the Tables from 1-4, while the engagement of the institution was given emphasison table 5 and the minimum yearly required points per faculty. The evaluation conducted focusedon both the engagement of each faculty and the required achievement of the institution's target
plans.
Figure (3) shows the different rates of performance assessment for each criterion. They vary inteaching and instruction standards from 36% to 48%, scholarly activity standards from 24% to25%, community service standards from 24% to 25%, and exceptional development standards
from 14% to 20%. Figures from (4-7) show the yearly faculty plan assessment for each category(superior, excellent and good) and the total average assessment. The highest evaluation of thefaculty was in community service standards, while the lowest one was in scholarly activity
standards. Figure (8) shows the yearly faculty plan total Assessment for each criterion and thetotal average assessment. Figures (9-123) show the annual faculty member's qualitative andquantitative performance per each category and criterion (overall faculty average was 68%). The
quantitative performance represents the faculty member degree of commitment, which reflects
the FIP achievement degree (faculty average is 78%), while the qualitative performance
represents the faculty member’s degree of excellence, which reflects the work quality degree(faculty average is 87%), as calculated in Tables 1-4. Figure (13) shows the total yearly faculty
member’s qualitative and quantitative performance. Figure (14) shows the faculty member's
continuous development in 5-years per each criterion and category.
Implementing the TBE Model equations will help the quality dimensions of the institutional
higher education as follows:
Faculty Members
From the analysis of the model, faculty evaluation has a formative purpose that the results areused to help faculty development, self-improvement and growth, and personnel decisions on
promotion and incentives.
Teaching Quality
From the statistical analysis module, faculty teaching performance will be improved specificityof feedback and alignment of assessments with the students’ assessment survey. As mentioned
in the activities plan of teaching and scholarly, faculty take advantage of the new technologicaltools to enhance student-to-faculty interaction and to better assess student progress. They connect
with advanced teaching practices to improve their teaching materials and methods. Faculty takethe opportunity to reflect on their own actions and role in the enhancement of teaching quality,
obtaining a commitment to reflective practice and causing adaptation and innovation.

42

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Figure 3: Rates of performance assessment

Figure 4: Faculty plan teaching assessment(per year)

Figure 5: Faculty plan scholarly assessment(per year)

Figure 6: Faculty plan service assessment(per year)

Figure 7: Faculty plan exceptional assessment(per year)

Figure 8: Faculty plan total assessment(per year)

43

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Figure 9: Faculty plan teaching assessment(quantity-quality)

Figure 10: Faculty plan scholarly assessment(quantity-quality)

Figure 11: Faculty plan service assessment(quantity-quality)

Figure 12: Faculty plan exceptional assessment(quantity-quality)

Figure 13: Faculty plan total assessment(quantity-quality)

Figure 14: Faculty plan development chart in
5-Years (just for verification)

44

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Teaching Competency Criteria
In addition to the faculty evaluation criteria by comparing equations (5) and (6), the model helps
in having well-aligned information inside the different levels of the institution (university,
department and faculty). The institutional teaching competency considering self-evaluations andlearner assessment data are combined to guide programming for faculty development, which will
help to improve the higher education performance in figuring out the following different criteria:- Average program/department assessment- Assessment comparison of all programs- Average assessment for all department- Top 5-faculty in the program/department- Best program in the department- Faculty/program/department continuous learning over time
- Institution development outcomes progress over time
- Average teaching/scholarly/service/creativity assessment per program/department- Average teaching/scholarly/service/creativity continuous learning over time perprogram/department
Institution Strategic Decision-Making
Over the above, the model helps in supporting the following strategic decision-making of theinstitution:- Strengths and weaknesses of program/department- Faculty training needs analysis- Institution strategic plan updating- Faculty workload matrix- Potentials for improvement plans- Faculty promotion plans- Building mechanisms to support faculty- Continuous improvement quality circles
Students
Applying the TBE Model will help the students to collaborate actively with faculty in thedefinition of the initiative and of the quality of the teaching concept itself. The model will assist
in keeping the interaction active and increasing concerns about teaching, learning environments,
quality of content and faculty attitudes.
Conclusions
There is a worldwide interest in developing quantitative faculty member's activity evaluation
models (Mitchell & Leachman, 2015). However, implementing a fair and reliable model ischallenging. There are numerous existing different evaluation models for faculty members. The
main objective of the current research is to propose a practical, comprehensive and flexible
statistical Target-Based Engagement (TBE) assessment model of faculty members. The model
considers both the specific faculty needs through selecting the matching activities in the faculty
member’s plan and the academic unit management concerns through providing a list of activities’references compatible with the unit strategic plan. The model involves several sequential phases
and has a significant impact on enhancing faculty performance and institutional quality. The
TBE model processing reliability was developed in the Department of Engineering andTechnology at Southern Utah University. The statistical model helps improving faculty

45

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

performance and is considered a long-term improvement method in the academic profession.
The consequences of this model will enhance the performance of higher education stakeholders.
The main components of TBE are:
1. Department strategic plan.
2. Department’s Activity Performance Standard Database (APSD) classified into four main
criteria: teaching and instruction standards, scholarly activity standards, community
service standards, and exceptional development standards. Each criterion has three value
categories (superior, excellent and good) with different activities worth.
3. Measurement Module (MM) that includes different methods for evaluating the faculty
member's performance from various sources, such as department chair, self-assessment,
students, peers, mentors, etc.
4. Faculty Improvement Plan (FIP) coordinated with the department’s strategic plan.
5. Statistical Analysis Module (SAM) that automatically calculates all different levels of
the university, department, and faculty by using MatLab R2019a.
Recommendations
The current research focuses on the determination of the reliability and validity of the suggested
TBE model. Based on the gap analysis of the literature, the model does not describe the findingsof the research study in detail, but rather it is representing the model design innovation,
development, concept orientation and potential application of a new faculty assessment model.
The application gives lesser importance as compared to the model development due to theimplementation time limitation that requires collecting data for five years to establish the whole
list of the statistical analysis aspects that the model could provide. In this paper, theimplementation of the model was based on one year of collected data. Subsequent studies must
include a more extended implementation period in order to determine the generalizability of themodel.

46

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
Agbetsiafa, D. (2010). Evaluating effective teaching in college-level economics using student
ratings of instruction: A factor analytic approach. Journal of College Teaching &Learning, 7(5), 57–66. https://doi.org/10.19030/tlc.v7i5.7841
Aho, K., Derryberry, D., & Peterson, T. (2014). Model selection for ecologists: The
worldviews of AIC and BIC. Ecology, 95, 631–636. https://doi.org/10.1890/13-1452.1
Apodaca, P., & Grad, H. (2005). The dimensionality of student ratings of teaching –Integration of uni-and multidimensional models. Studies in Higher Education, 30(6),723–748. https://doi.org/10.1080/03075070500340101
Arlot, S., & Celisse, A. (2010). A survey of cross-validation procedures for the model
selection. Statist. Surv. 4, 40–79. https://doi.org/10.1214/09-SS054
Banta, T. W., & Associates. (2002). Building a scholarship for assessment. San Francisco, CA:
Jossey-Bass Publishers.
Chang, R. M., Kauffman, J., & Kwon, Y. (2014). Understanding the paradigm shift tocomputational social science in the presence of big data. Decision Support Systems. 63,
67–80. https://doi.org/10.1016/j.dss.2013.08.008
Chen, Y., & Hoshower, L. B. (2003). Student evaluation of teaching effectiveness – Anassessment of student perception and motivation. Assessment & Evaluation in Higher
Education, 28(1), 71–88. https://doi.org/10.1080/02602930301683
Dehmer, M. & Emmert-Streib, F. (2017). Frontiers Data Science; CRC Press: Boca Raton.
FL, USA.
Ding, J., Tarokh, V. & Yang, Y. (2018). Model selection techniques: An overview. IEEE
Signal Processing Magazine 35, 16–34.
Dougherty, K. J., & Natow, R. S. (2015). The performance politics funding for higher
education: Origins, discontinuations, and transformations. Baltimore, MD: JHU Press.
Ellis, R., & Bowden, R. (2014). Performance-based funding: Changing the paradigm for higher
education. British Journal of Education, Society & Behavioral Sciences, 4(7), 942–952.
https://doi.org/10.9734/BJESBS/2014/9236
Elmore, H. W. (2008). Towards Objectivity in faculty evaluation. Academe, 94(3): 38–40.
Emmert-Streib, F., & Dehmer, M. (2019). Defining data science by quantification of datadriven of the community. Machine Learning and Knowledge Extraction, 1, 235–251.
https://doi.org/10.3390/make1010015
Emmert-Streib, F., & Dehmer, M. (2019). High-dimensional LASSO-based computational
regression models: Regularization, shrinkage, and selection. Machine Learning andKnowledge Extraction, 1, 359–383. https://doi.org/10.3390/make1010021
Emmert-Streib, F., Moutari, S., & Dehmer, M. (2016). The process of analyzing data is an
emergent feature of data science. Frontiers in Genetics, 7(12).
https://doi.org/10.3389/fgene.2016.00012
Forster, M. R. (2000). Key concepts in model selection: Performance and generalizability.
Journal of Mathematical Psychology, 44, 205–231.
https://doi.org/10.1006/jmps.1999.1284

47

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Grunwald, H., & Peterson, M. W. (2003). Factors that promote faculty involvement in andsatisfaction with institutional and classroom student assessment. Research in Higher
Education, 44(2), 173–204. https://doi.org/10.1023/A:1022051728874
Hardin, J., Hoerl, R., Horton, J., Nolan, D., Baumer, B., Hall-Holt, O., … Ward, M. D. (2015).
Data science in statistics curricula: Preparing students to ”think with data”. The
American Statistician, 69, 343–353. https://doi.org/10.1080/00031305.2015.1077729
Havnes, A. & McDowell, L. (2008). Introduction: Assessment dilemmas in contemporary
learning cultures. In A. Haynes & L. McDowell (Eds.), Balancing dilemmas inassessment and learning in contemporary education, (pp. 3–14). Routledge: New York.
Hillman, N. W., Tandberg, D. A., & Fryar, A. H. (2015). Evaluating the impacts of the “new”
performance funding in higher education. Educational Evaluation and Policy Analysis,
37(4), 501–519. https://doi.org/10.3102/0162373714560224
Fletcher, R. B., Meyer, L. H., Anderson, H., Johnston, P., & Rees, M. (2012). Faculty andstudents conceptions of assessment in higher education. Higher Education, 64, 119–133. https://doi.org/10.1007/s10734-011-9484-1
Kuh, G. and Ikenberry, S. (2009). More than you think, less than we need: Learning outcomes
assessment in American higher education. National Institute for Learning Outcomes
Assessment.
Maki, P. L. (2010). Building a sustainable commitment across the institution: Assessing forlearning (2nd ed.). Sterling, VA: Stylus Publishing.
Miao, K. (2012). Performance-based funding of higher education – A detailed look at bestpractices in 6 states. Washington, DC: Center for American Progress.
Mitchell, M., & Leachman, M. (2015). Years of cuts threaten to put the college out of reach formore students. Washington, DC: The Center on Budget and Policy Priorities.
Mittal, S., & Gera, R. (2013). Student evaluation of teaching effectiveness (SET): An SEM
study in higher education in India. International Journal of Business & Social Science,4(10), 289–298.
O’Shaughnessy, L. (2013). 6 Challenges facing state universities. CBSNews.
Palomba, C. A., & Banta, T. W. (2015). Assessment essentials: Planning, implementing andimproving assessment in higher education. San Francisco, CA: Jossey-Bass Publishers.
Provost, F., & Fawcett, T. (2013). Data science and its relationship to big data and data-driven
decision-making. Big Data, 1, 51–59. https://doi.org/10.1089/big.2013.1508
Scheerens, J., & Bosker, R. J. (1997). The foundations of educational effectiveness. Oxford,
UK: Pergamon.
Seidel, T., & Shavelson, R. J. (2007). Teaching effectiveness research in the past decade - Therole of theory and research design in disentangling meta-analysis results. Review of
Educational Research, 77(4), 454–499. https://doi.org/10.3102/0034654307310317
Stufflebeam, D., L. (2003). The CIPP Model for Evaluation. The Annual Conference ofOregon Program Evaluators Network, Portland, Oregon.
Umbricht, M. R., Fernandez, F., & Ortagus, J. C. (2017). An examination of the (Un)intended
consequences of performance funding in higher education. Educational Policy, 31(5),643–673. https://doi.org/10.1177/0895904815614398

48

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Wit, E., van der Heuvel, E., & Romeijn, J.W. (2012) ”All models are wrong...”: An
introduction to model uncertainty. Statistica Neerlandica, 66(3), 217–236.
10.1111/j.1467-9574.2012.00530.x
Zucchini, W. (2000). An introduction to model selection. Journal of Mathematical Psychology,
44(1), 41–61. https://doi.org/10.1006/jmps.1999.1276
Corresponding author: Mohamed Askar
Contact email: mohamedaskar@suu.edu

49

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 201950

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

E-learning Self-efficacy of Operating Room Nurses of a Selected Hospitalin Cebu, Philippines
Paul John G. Aventurado
Cebu Doctors’ University, Philippines

51

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
The conventional methods of education and training for operating room experience have been
rendered impractical due to both the shortage and fast turnover of nurses in hospitals. Experts
in the field of education are promoting e-learning modalities as a potential solution. However,issues were raised regarding the readiness of nurses to engage in e-learning due to lack ofstandardized competencies for practice, inconsistency in the integration of informatics courses
in education, and lack of experts in the field. Literature reveals that self-efficacy is a major
component of e-learning readiness. A descriptive-survey design was utilized to assess the elearning self-efficacy of operating room nurses in a selected hospital in the Philippines. The
Modified E-learning Readiness Assessment Tool (MERAT) captured the elements of theconstruct along with its sub-domains: computer, internet/online, and software utilization selfefficacy. The frequency distribution and item means were computed. Item analysis revealed
that the respondents have positive e-learning self-efficacy based on item means obtained
ranging from 3.29 to 4.58 on a 1.0 to 5.0 scale. A secondary analysis was conducted byclustering the e-learning self-efficacy indicators based on the nature of the competency anddata trends. It revealed that the participants perceived themselves to be “Very Good” in termsof complex computer skills, online etiquette, troubleshooting computer-related issues, andusage of an e-learning platform, and “Excellent” in terms of basic computer and online skills.
The results of this study implied that e-learning initiatives might be used to augment educationand training of operating room nurses despite issues on manpower and competency base. It isfurther recommended that research explore other domains of e-learning readiness anddetermine the impact of this modality in the field of health care.
Keywords: e-learning, education, training, nurses, operating room, self-efficacy

52

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Introduction
A global threat to the healthcare system is emerging as the demand for quality care increases
with depleting medical resources. This scenario is exemplified with the shortage and fast
turnover of nurses in hospitals, which negatively impacts the transition of knowledge and skills
especially in areas that require highly technical competencies such as the operating room
(Littlejohn, Campbell, Collins-McNeil, & Thembisile, 2012; Momanyi & Kaimenyi, 2015;
Takase, 2010). This imbalance of supply and demand must be perceived and addressed withurgency and salience by the health sector in society. Training and education of medical
practitioners have become more difficult as the context of health care becomes moredemanding. The conventional preceptor-based method of education and training of nurses may
not be sufficient for them to provide quality care in the face of heavy workloads, rigid
schedules, and the low ratio of experienced to inexperienced personnel. The current scenario
creates an opportunity for Information and Communication Technologies (ICT) and methods
such as e-learning into the field of surgical training to showcase its potential and advantages.
Studies have shown positive outcomes from e-learning utilization (Kadivar, Seyedfatemi,
Zolfaghari, Mehran, & Hossinzade, 2016; Maertens et al., 2016), but concerns arise regarding
the readiness of nurses to engage in e-learning. This stems from the lack of standardized
competencies for practice, inconsistency in the integration of nursing informatics in education,and lack of experts in the field (Staggers, Gassert, & Curran, 2001). Nurses must be ready toembark on this new modality of learning as failure to do so will likely lead to the failure of elearning initiatives (Demir & Yurdugűl, 2015). E-learning readiness was conceptualized byscholars to be multifaceted with self-efficacy one of its major components.
This study assessed the e-learning self-efficacy of operating room nurses of a selected hospitalin Cebu, Philippines. Findings from this study shed light on the feasibility and potential of
initiating e-learning modalities in the field of nursing to augment education and training in anenvironment of high demands and decreased resources.
Literature Review
The nursing labor force is the front liner of the healthcare system, yet there is nursing shortage
on both local and global frontiers. The dearth of healthcare providers is recognized by agencies
such as the International Council of Nurses (2004) and the High Level Forum on HealthMillennium Development Goals (2004) and is a cause of concern for health delivery andoutcomes. Littlejohn et al. (2012) delved into the nursing shortage in the US, Philippines andSouth Africa and reported that this setback in manpower exists in developed and third-world
countries with different contexts, but with similar adverse effects to the health care system. The
nature of the nursing shortage in South Africa due to migration, limited opportunities for
employment, and attrition to diseases will not be highlighted in this paper. The shortage in the
United States is due to an aging workforce and lack of faculty in nursing schools. This gap intheir manpower is filled up by countries like the Philippines. The country is a top exporter ofnurses overseas which is primarily intended to ease labor market concerns. However, this leads
to repercussions as well-trained and skilled nurses in the country leave faster than they can be
replaced. Statistics in 2010 show that 85% of Filipino nurses seek work abroad. Migrating andexperienced nurses leave a knowledge and skills void which is associated to the “brain drain”
phenomenon in the country. The International Labor Organization (2006) conceded that mostof these professionals who migrate are highly skilled and are hard to replace. The cycle
continues as more nurses leave to seek better opportunities in industrialized countries withaging populations. This translates to a rapid turnover at the hospitals particularly in specialized53

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

units like the operating room where highly specialized and technical skills are required toprovide quality care. Littlejohn et al. (2012) recommended actions to address these issues in
the global health force by providing better practice environments, increasing financial
compensation and improved policies on recruitment and retention. However, despite a wider
and comparative view of the global nursing shortage, no offer was given regarding specificmeasures or action plans to deal with the ongoing issues which potentially threaten the futureof health care.
A fast nurse turnover at the hospitals may aggravate the current nursing shortage and loss of
trained human capital (Takase, 2010; Momanyi & Kaimenyi, 2015). This perspective hasturned into reality in the current setting of tertiary hospitals in the Philippines. To date, Cebu
Doctors’ University Hospital is serviced by less than 40 registered nurses to cover 8 operating
room theaters that are utilized to provide surgical services on a 24-hour basis. Nurses newly
assigned to the unit reported that they undergo preceptorship before being assigned to scrub or
circulate in surgical cases. These nurses further claimed that the ratio of experienced to thenovice nurses is too low to warrant good outcomes from this type of training. Heavy workloads
and rigid schedules further make preceptorship impractical. This insider perspective of thehospital mirrors the report of Littlejohn et al. (2012) which described the movement of nurses
from third-world countries to favor the developed ones. This renders the institution heavily
disadvantaged in terms of manpower. Furthermore, poor return of investment in training andeducation is expected as workers are retained on a short-term basis and migrate to othersettings. An alternative mode of training and education is needed to transmit knowledge andskills in the midst of low manpower, decreased time allocation for training and increased
healthcare demands.
Training and education of medical practitioners have become more difficult as the context ofhealth care becomes more demanding. The incongruence between the supply of skilledworkforce and demand for quality care fast-tracked the ushering of Information andCommunication Technologies (ICT) and methods such as e-learning into the field of surgicaltraining. The conventional preceptor-based transition of knowledge and skills may not be apragmatic method of surgical training due to rapid nurse turnover and demand for manpower.
Kadivar et al. (2016) contended in their paper that nurses have high client loads and rotational
shifts which make traditional education impractical; using e-learning, on the other hand, candecrease learning time by 30 to 35%. Further studies however are needed to specificallydetermine the value of e-learning as primary or supplemental modality of education andtraining. Some scholars report that this new modality may have similar effectiveness withconventional and non e-learning based interventions, but nonetheless, e-learning is at least as
effective as other methods (Maertens et al., 2016).
Several scholars have appraised and recognized the value of e-learning in the field of surgicaleducation and training. The advantages of e-learning such as accessibility, flexibility, and lowcost implementation, complement the “highly procedural” nature of surgery (Maertens et al.,2016). Competency in the surgical field involves psychomotor, cognitive and non-technicalelements. Examples of technical psychomotor skills include operating room-specificproficiency in instrumentation (use or assist) and knot-tying. Cognitive elements include
knowledge about the procedure or disease pattern recognition. The non-technical aspectincludes generic, but important, characteristics such as communication, leadership, andprofessionalism. Overall, it is recognized that e-learning has the capacity to develop surgicalcompetencies due to the mechanics of repetition, focused practice, and immediate feedback.

54

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Maertens et al. (2016) posited that e-learning is an effective platform to develop surgicalcompetencies. Its potential is best when combined with other modes of learning as it provides
the impetus at the beginning of surgical skills development, before a degree of automaticity isachieved in clinical exposure to the operating room. The capacity to review the e-learningmaterial also lessens the performance decay of the personnel. As an added effect, Momanyi
and Kaimenyi (2015) presented from their investigation that exposing nurses to good training
programs and continuing education may help decrease nurse turnover as it develops
competencies and enhances morale and efficiency. These reports revealed a mechanism on
how e-learning addresses manpower issues through retention of nurses. It operates on thepresumption that nurses will stay longer in their line of work if they feel fulfilled in their roles
as competent health care providers. In this perspective, e-learning may not be the prime mover
but instead, a supplemental factor that influences deters fast nurse turnover.
The diffusion of e-learning into the surgical training of staff can indeed help augment the
impact of high nurse turnover at the hospitals. However, several authors (Oliver, 2001; Akaslan& Law, 2011; Moftakhari, 2013, as cited in Demir & Yurdugűl, 2015) reiterated the need forusers to be ready to engage in e-learning before embarking on this new modality. One major
reason for the failure of e-learning initiatives to thrive is the lack of readiness (Demir &Yurdugűl, 2015). Confidence to handle e-learning materials among nurses may not be fully
developed due to lack of standardized competencies, inconsistency in the integration of
Nursing Informatics (NI) in the curricula and lack of nursing experts in the field (Staggers,
Gassert, & Curran, 2001). These deficiencies were highlighted and associated to e-learning
readiness as it hinders the capability of nurses to fully benefit and utilize the resource. The
proponent of this paper, being an educator in nursing informatics, also notes that these are valid
concerns in the current educational set-up in the country.
Several healthcare leaders advocate for incorporating informatics competencies for nurses in
the academe and clinical setting. However, no consensus was reached regarding the ideal
informatics competencies to embody, and even local initiatives are slow to progress. Agencies
such as the American Association of Colleges for Nursing (AACN), American Nurses
Association (ANA), National League for Nursing (NLN) and the International Medical
Informatics Association (IMIA) have published relevant papers on NI competencies but to noavail in developing a consistent list of competencies. IMIA released a set of NI competencies
but these were not scientifically validated. Scholars such as Grobe (1989, as cited in Staggers,
Gassert, & Curran, 2001) expressed the need for a current and validated list of NI competencies
as a guiding framework for curricular development as this aids in preparing the nurses for theirroles in practice and to manage expectations of potential employers and health care consumers.
Staggers et al. (2001) spearheaded a movement to create a valid and reliable set of NI
competencies. It involved extracting competencies from existing literature and subjecting these
for appraisal by a team of experts. A list of competencies was presented which included
competencies in computer skills (data entry and access, use of email), informatics knowledge(ethical and practical use of data), and informatics skills (troubleshooting, screen layout) alongfour different skill levels. The most basic level, beginning nurse, has fundamental knowledge
and skills in information management and computer technology. An experienced nurse isproficient in a chosen domain (education, administration) and can utilize data set and trends
for productivity. An informatics specialist has advanced preparation and can use moresophisticated applications including systems development. An informatics innovator represents
the highest level, and portrays one who can utilize research and generate theory in the field.
There were more scholarly movements that led to the inception of other NI frameworks, but

55

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

this paper is anchored in the work of Staggers et al. (2001) as this is a pioneering empiricalbased and validated model and one of the foundations of locally offered informatics courses
familiar to the proponent of this paper.
Bernal, Tolentino, Gavino, Fontelo, and Marcelo (2008) discovered that nursing schools in thePhilippines approach the field of NI in a superficial manner. It focuses only on basic computer
skills or on applications which are expected to be basic in an environment of technologicalinflux. The proponent’s affiliated institution, as an example, offers the nursing informatics
course in the bachelor’s program for two semesters. The first segment of the course deals withbasic computer skills such as word processing and spreadsheet application. Database concepts
and applications of informatics to education, clinical care and research are given subsequently.
Graduates of the program will still be far from the highest level of competency (Staggers et al.,2001) which involves capabilities to use sophisticated software, design and innovation.
Nursing in the country is geared more towards delivery of health services to clients in a clinicalor community setting. This is probably one reason why the Philippines is one of the highly
favored exporters of nursing manpower (Littlejohn et al., 2012) as developed countries seek
care providers at the bedside rather than informaticists. Other concerns in incorporating NI
competencies in the country include the lack of a locally appropriate model for NI practice anda small number of experts in the field. This limits the training ground and exposure of potential
nurse informaticists for developing their potential. Bernal et al. (2008) suggested the following
actions to address the NI deficiencies in the country: evaluating NI models for practicalapplication in the local arena, enhancing post graduate training, and increasing awareness for
NI. The concerns relayed in literature may be reasons for poor utilization of e-learning as ameans of surgical training despite its effectiveness. It is important, therefore, to assess thereadiness of nurses to engage in e-learning before initiatives are started.
A state of readiness is achieved when users “believe” that they can optimize the use of elearning platforms to attain expected outcomes. E-learning readiness is important for users to
be able to optimize the use and attain benefits form online platforms. Demir and Yurdugűl
(2015) investigated the most comprehensive and prevalent models of e-learning readinessavailable in literature with the aim of developing a current and validated model. They foundout that most models were developed to suit different stakeholders, including learners, teachers
and institutions. Common components of these frameworks were the belief in one’s
competency in technology usage and confidence in prerequisite skills for e-learning. Thisfeature of e-learning readiness is equitable to self-efficacy. The model of Hung, Chou, Chen,
and Own (2010) for example presented five components including computer and internet selfefficacy. Dray, Lowenthal, Miszkiewicz, Ruiz-Primo, and Marczynski (2011) named belief intechnology skills as a dimension of learner self-efficacy. Smith (2005) tested the scale
developed by McVay (2000) about e-learning readiness which included comfort with elearning as one of its factors. Tubaishat and Lansari (2011) developed a scale which included
the confidence and perception of e-learning among students as a dimension. Valtonen,
Kukkonen, Dillon, and Vaisanen (2009) presented beliefs in online learning and ICT skills as
a dimension of readiness for online learning. These are some of the models reviewed by Demirand Yurdugűl (2015). There were also other prominent dimensions of e-learning readinesstackled by other scholars such as personal factors (motivation, preference, time management)
and infrastructure which is not covered in this paper.
Self-efficacy research started in the 1970s, but it was not until 2008 that online self-efficacy
came into focus. A major proponent of self-efficacy theory is Bandura (1997) who defined theterm as the belief in one's capability to execute a task. This belief further influences a person's

56

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

choice, action, and satisfaction. Four major sources of information that determines self-efficacy
include (1) performance accomplishments or inactive mastery experience which refers tovalidation of one's success in doing a task, (2) vicarious experience, inspiration from otherpeople's capacity to perform the same task, (3) verbal persuasion referring to authentic
feedback of achievement and (4) physiologic states as reflected by anxiety and stress involvedin doing a task. The author of this paper initially was reluctant to ground this study on thetheory of Bandura as the e-learning concept did not materialize yet during the early
development of the self-efficacy theory. There were, however, studies which bridged themacro-level theory of Bandura to the recent concept of e-learning. A study by Lin, Liang,
Yang, and Tsai (2013, as cited in Alqurashi, 2016) showed that the general concept of Bandura(1997) on self-efficacy is applicable among adult online learners. Embedding the concept ofBandura (1997) in the context of e-learning, a person with high self-efficacy has positive selfappraisal and recognizes the challenges of learning through an online environment. They seek
available resources to fully utilize it and gain a sense of personal accomplishment. A personwith low self-efficacy has low belief in self and will avoid engaging with the new mode oflearning and deny diffusion of technology into practice. They linger on their failure andeventually disengage in e-learning.
Other studies were conducted and delved deeper into the context of self-efficacy in online
learning. Alqurashi (2016) conducted a literature review of studies about online learning selfefficacy conducted between the years 1997 to 2015. Three major themes emerged as influential
factors that determine online self-efficacy: (1) Computer Self-efficacy – utilization of computerand related technology, (2) Internet and Information Seeking Self-efficacy – navigation andaccessing suitable information over the World Wide Web and (3) Learning Management
System (LMS) Self-efficacy – interaction with the functionalities of a LMS platform.
Computer self-efficacy refers to the confidence in using the computer and related technology.
Alqurashi (2016) presented several studies such as those of Jan (2015), Lim (2001), Pellas(2014), Simmering (2009), and Womble (2007) which pointed out computer self-efficacy as asignificant factor for e-learning. Internet and information seeking self-efficacy relates to thecapability of navigating through the World Wide Web with confidence. The proponent also
found several studies confirming the association between internet self-efficacy and e-learningwhich included those of Kuo, Walker, Schroder, & Belland, (2014), Tang and Tseng (2013),
and Womble (2007). LMS self-efficacy refers to the confidence of users to utilize different
functionalities of an e-learning platform such as asynchronous and synchronous
communication, accessing course content and advanced tools. This component was found tohave a positive impact on course performance in an online environment. This paper
conceptualized e-learning self-efficacy based on these published works as the nurses’ trust in
their ability to succeed in e-learning through proficiency in using computer-related technology,
online navigation, and platform utilization.
The mandate to incorporate e-learning competencies in the Philippine curriculum did notconcretely materialize until 2012 through the Commission on Higher Education (CHED)
Memorandum Order 46. With the said timeline, it is safe to say that e-learning is still in theinfancy stage in the country. This supported the notion of Doculan (2016) that most research
instruments on e-learning were developed for “e-mature” countries. Models that have been
proposed to assess the degree of e-learning readiness such as those of Aydin and Tasci (2005),
Chapnick (2000), and Watkins (2004) were not meant for developing countries. She further
stated that e-learning models may not universally apply due to varying needs of stakeholders.
Research tools should consider the culture, norms and infrastructure of a country. The Modified
E-learning Readiness Assessment Tool (MERAT) was developed based on a Philippine study

57

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019assessing online self-efficacy. It addressed issues of compatibility and was created in thecontext of the local setting to capture the domain of e-learning self-efficacy of operating room
nurses working in the Philippines. Furthermore, efforts were made to validate the applicabilityof the items on the MERAT to the Philippine setting by appraisal from authorities in the fields
of computer and nursing. As it was developed within the local context, the questions on thetool cover only the first 2 levels of informatics competency presented by Staggers et al. (2001).
The MERAT may not be applicable for developed countries with high-level technology use.
The tool specifically included items relating to computer skills, internet, and online skills, andsoftware application, which aligned with the conceptualization of experts previously
mentioned. It was also intended to help in creating a comprehensive and sustainable e-learning
strategy for the country.
Major players in education emphasized the need for incorporating e-learning into the country’s
school system. Some of the e-learning initiatives have gained ground and were sustained
despite some limitations in infrastructure and user competencies. The University of thePhilippines established the UP Open University (UPOU). The University of Santo Tomas has
its own e-Learning Access Program (e-Leap). TESDA, a government agency, spearheaded theuse of Moodle, an online-based course management system that allows trainers to implement
e-learning programs. Arimbuyutan, Kim, Song, and So (2007) on the contrary, conceded that
Filipinos are used to the traditional classroom set-up, which may slow down the transition to a
new learning modality. They, however, saw potential from a large sector of Filipinos utilizing
the internet and engaged in online gaming. This may lead to a promising combination of users
and infrastructure. Fung (2016) further proclaimed the youths of today as “digital natives” whoare familiar with computer functionalities. These users are expected to harness the potential ofe-learning. The current cadre of nurses working in the hospitals falls into this generation. It
remains to be seen if this innate instinct to use technology outweighs the deficiencies of thenursing educational sector. Arimbuyutan et al. (2007) implored for the advancement of elearning in the Philippines to remain competitive in the global market. The community must
address issues towards this new frontier, particularly in infrastructure, investment, pedagogy,
cost, and resistance to change.
Methodology
Objective
This study sought to assess the e-learning self-efficacy of operating room nurses in a selected
hospital in Cebu, Philippines.
Research Design
A descriptive survey design was used to determine the e-learning self-efficacy of the
respondents of the study.
Research Environment
This study was conducted at the operating room unit of Cebu Doctors' University Hospital(CDUH). The institution is a tertiary unit hospital providing general to specialized care todifferent clienteles. CDUH-OR has 8 operating room theaters that cater to general andspecialized surgical cases and a special unit dedicated to eye surgeries. It also has a Post

58

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Anesthesia Care Unit (PACU) where clients are monitored after surgery. Data collection was
conducted in the conference room situated inside the unit.
Research Participants
A total of 31 operating room nurses were recruited and completed the survey during March
2019. The roster of nurses was obtained from the list provided by the nursing department.
Nurses who refused to participate, were on leave status during the data collection, and thosewith pending resignation were not included. The data collection package, which includes theinformed consent form and questionnaire were printed on paper prior to data collection. The
sheets were distributed and collected personally by the proponent to the respondents of thestudy. New nurses assigned to the unit relayed that they needed another avenue for learning as
the preceptorship they undergo during the training process is not comprehensive enough to deal
with more sophisticated and specialized surgical cases. The rigid schedule of nurses also makes
it difficult to participate in continuing education courses. They also utilize printed anddownloaded materials for additional learning.
Research Instrument
This study utilized the Modified E-learning Readiness Assessment Tool (MERAT). It is a 16item questionnaire that assesses for self-efficacy in using the computer, navigating the internet
and online environment, and utilization of software such as the Learning Management System(LMS) for e-learning. Each statement in the MERAT is matched to a 5-point Likert Scale
represented by 1 (Not at all), 2 (Very Least), 3 (Little), 4 (Great) and 5 (Very Great) whichrefers to the respondent’s confidence to perform the activity.
The items on the MERAT cover the indicators of e-learning self-efficacy as conceptualized inthis study, which includes: Computer Skills, Internet/Online Skills, and Software Application
Skills. Computer Skills Self-efficacy refers to the confidence in the utilization of computer andrelated technology. These include item numbers 1 (Knowledge on Basic Computer Function),
4 (Troubleshooting Hardware Problems), 14 (Utilization of Word Processing Software), 15(Multitasking in Multiple Workstations) and 16 (Utilization of Spreadsheet). Internet or Online
Skills Self-efficacy refer to confidence in accessing and navigation on the World Wide Web
and information-seeking capabilities over the net. These include item numbers 5 (EmailAttachment), 6 (Online Etiquette), 7 (Internet Navigation), 8 (Internet Browser Usage), 9(Troubleshooting Internet Connection Problem) and 13 (File Downloading). Software
Application Skills refer to the users' confidence in utilizing computer programs or applications.
There are 5 items related to the utilization of the e-learning platform, including item numbers
2 (Document Access), 3 (Configuration Setting), 10 (File Search and Download), 11 (Online
Library) and 12 (Asynchronous Tools).
The instrument also possessed good psychometric properties based on its administration to
respondents in Luzon and Cebu, Philippines. It has a Cronbach’s alpha ranging from 0.7 to
0.75.
Treatment of Data
Descriptive statistics were used to determine the e-learning self-efficacy of the respondents.
An item analysis was primarily conducted using percentage and mean computation. An
inductive process was made to identify patterns and clusters of indicators for a secondary59

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019analysis of the e-learning self-efficacy. The statistical software IBM SPSS version 22 was used
to process the data. Data were organized and presented through tables.
Ethics Review Process
The study protocol was submitted for initial review to Cebu Doctors’ University - Institutional
Ethics Review Committee (CDU-IERC) on February 20, 2019. The study qualified forexpedited review under the protocol code 2019-040-Aventurado-OperatingRoom. The
issuance of the CDU-IERC code signified fulfilment of the institutional requirement for
mandatory registration of the research study. The paper was subjected for appraisal by aprimary and secondary reviewer. The result of the first review by the panel was released
February 26, 2019 with minor modifications required prior to approval. Revisions were madeand the study was granted approval for implementation February 27, 2019. The CDU-IERC
Final Report was accomplished by the proponent at the completion of the study and ethicalclearance was secured on April 8, 2019.
The study was conducted in compliance with the prescribed institutional guidelines for ethics
which specifically included the following: (1) Informed consent was obtained from therespondents which signified their understanding of the nature of the study, voluntary
participation and willingness to partake the benefits from the study; (2) study protocols suchas controlled room entry and appropriate spacing were strictly adhered during data collection
to ensure protection of the participants’ rights and integrity; and (3) data obtained from thestudy were collected and kept with confidentiality through removal of identifiers and using ofcode numbers.
Findings
Item Analysis
An item analysis of the e-learning self-efficacy of the respondents was conducted based on the
Modified E-learning Readiness Assessment Tool (MERAT). Presented below are the items on
the MERAT on the first column, followed by the distribution of responses according topercentage. Item mean was also computed to describe the scale and provide a basis of indicator
clustering for further analysis.
Item mean interpretation was based on the following hypothetical ranges and descriptors: 4.215.00 (E – Excellent), 3.41-4.20 (VG – Very Good), 2.61-3.40 (Good), 1.81-2.60 (Satisfactory),
1.00-1.8 (Poor). The table below presents the item analysis of each indicator of e-learning selfefficacy among the operating room nurses of the selected institution.

60

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 1: E-learning self-efficacy item analysis
Not at
all

Very
least

Little

Great

Very
Great

Item
Mean

Intr.

0%

0%

3.2%

41.9%

54.8%

4.52

E

0%

9.7%

22.6%

48.4%

19.4%

3.77

VG

0%

12.9%

16.1%

58.1%

12.9%

3.71

VG

0%

9.7%

29%

45.2%

16.1%

3.68

VG

0%

3.2%

3.2%

41.9%

51.6%

4.42

E

0%

6.5%

22.6%

61.3%

9.7%

3.74

VG

7. I know how to surf the internet andnavigate the web.

0%

0%

3.2%

38.7%

58.1%

4.55

E

8. I can use web browsers (Internet
Explorer, Google Chrome) confidently.

0%

0%

3.2%

45.2%

51.6%

4.48

E

0%

12.9%

25.8%

35.5%

25.8%

3.74

VG

0%

12.9%

25.8%

41.9%

19.4%

3.68

VG

0%

6.5%

25.8%

41.9%

25.8%

3.87

VG

0%

12.9%

19.4%

51.6%

16.1%

3.71

VG

13. I know what PDF files are and I candownload and view them.

0%

0%

3.2%

48.4%

48.4%

4.45

E

14. I am comfortable with word
processing and use it comfortably.

0%

0%

6.5%

29%

64.5%

4.58

E

0%

0%

6.5%

35.5%

58.1%

4.52

E

3.2%

12.9%

41.9%

35.5%

6.5%

3.29

VG

Indicators
1. I know the basic functions of computer
hardware components (CPU, monitor)
including its peripherals like printer,speaker and mouse.
2. I know how to open documents in an
e-learning platform.
3. I am comfortable in changing theconfiguration settings of an e-learning
platform.
4. I know how to resolve common
computer hardware or software problems
or I can access technical support in case I
encounter a problem.
5. I can send an email with fileattachments.
6. I am familiar with online etiquette.

9. I know how to resolve common errors
while surfing the internet like “pagecannot be found” or “connection timeout”.
10. I am comfortable with things likedoing searches and downloading files inan e-learning platform.
11. I know how to access an online
library in an e-learning platform.
12. I know how to use asynchronous
tools in an e-learning platform likediscussion boards and chat tools
effectively.

15. I am able to open several applications
at the same time and move between
them.
16. I know how to use spreadsheet
applications.

61

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Most of the participants’ responses were positive (great or very great) among all the items of
MERAT except item 16. Most of the respondents answered “great” for items 2, 3, 4, 6, 9, 10,
11, 12 and 13. Most of the respondents answered “very great” for items 1, 5, 7, 8, 13, 14 and15. The highest item mean was computed for item number 14 which reflects utilization of word
processing software, and item number 7 which depicts capability in internet navigation. The
lowest item mean was obtained for item 16 which reflects spreadsheet mastery, item 4 which
involves troubleshooting hardware issues and item 10 which reflects ability to search anddownload files. The participants obtained mean score ranges of 3.29 to 4.55, with nine out of
16 indicators interpreted as “Very Good” self-efficacy and 7 indicators rated as “Excellent” inself-efficacy.
Indicator-Clusters Analysis
The researcher analyzed the results of the item analysis and synthesized the individual
indicators of e-learning self-efficacy into clusters. Clustering was done by utilizing theprinciple of similarity and contrast while considering the data trends within a set of indicators.
For example, indicators that point to basic computer competencies may be grouped together,but may be subdivided based on the complexity or resources required to perform the skill. The
different themes that emerged from clustering are presented on the table below.
Table 2: E-learning self-efficacy indicator-clusters analysis
Cluster

Description

A

Basic Computer Skills

B

Complex Computer
SkillsC

Basic Online Skills

D

Online EtiquetteE

TroubleshootingFE-learning Platform

Set of Indicators
Basic Computer Function, Word Processing,Spreadsheet Application
Troubleshooting, Spreadsheet Application
Email Attachment, Internet Navigation, Browser
Utilization, File Downloading
Online Etiquette
Troubleshooting: Hardware and Software, Internet
Connection
E-learning Platform: Document Access,Configuration, File Search and Download, Online
Library, Asynchronous Tools

Interpretation
E
VG
E
VG
VG
VG

The indicators relating to computer skill self-efficacy are items 1, 4, 14, 15 and 16. Participants
rated items 1 (Basic Computer Function), 14 (Word Processing) and 15 (Multiple
Workstations) with “Excellent” self-efficacy. These indicators are labeled as Cluster A and areconsidered as very basic computer skills. Cluster B are indicators related to computer skills but
are more complex and not within the entry-level domain of nursing. This includes items 4(Troubleshooting: Hardware and Software) and 16 (Spreadsheet Utilization) which obtained arating of “Very Good” self-efficacy among the respondents.
The indicators relating to internet and online self-efficacy were items 5, 6, 7, 8, 9 and 13.
Cluster C consolidated items 5 (Email Attachment), 7 (Internet Navigation), 8 (Browser
Utilization) and 13 (File Downloading). These online skills are the basic ones and are part ofregular internet activity which justified the “Excellent” self-efficacy rating of the respondents
to these items. Item 6 was placed on a separate cluster due to its unique nature and lower rating
among its counterpart. Cluster D represents online etiquette which obtained a mean item
interpretation of “Very Good”.

62

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Cluster E grouped items number 4 and 9 which shares the domain of troubleshooting hardware
and internet issues. The last set of indicators, Cluster F, relate to software application selfefficacy which include item 2 (Document Access), 3 (Configuration Setting), 10 (File Searchand Download), 11 (Online Library) and 12 (Asynchronous Tools). Respondents rated
themselves “Very Good” in terms of self-efficacy in Clusters E and F.
Clustering of the indicators based on their similarity and data trends is a preliminary step todetermine the need for improvement. Grouping together similar indicators economicallyallowed for more specific measures to address the concern. This subsection culminates withthe categorization of the indicators for e-learning self-efficacy into clusters: Cluster A (Basic
Computer Skills), Cluster B (Complex Computer Skills), Cluster C (Basic Online Skills),
Cluster D (Online Etiquette), Cluster E (Troubleshooting) and Cluster F (Utilization of Elearning Platform).
Discussion
The findings of this study revealed that participants have acceptable levels of e-learning selfefficacy with the majority of the indicators rated positively based on item responses and means.
This contradicts the prior assumption that nurses are not prepared to engage in e-learning due
to lack of standardized competencies, inconsistency in the integration of Nursing Informatics
in the curricula and lack of nursing experts in the field (Staggers, Gassert, & Curran, 2001).
The operating room nurses of the selected hospital reflect individuals with good self-efficacy
who recognize the challenges and seek the full potential of online learning (Bandura, 1997).
This tendency has been attributed in part to the younger generation of nurses getting into theworkforce who have been educated in nursing informatics in the newly revised nursing
curriculum. They may have gained competencies from the nursing educational program prior
to their nursing career.
A look into the history of nursing informatics in the Philippines revealed that despite the lackof standardized model and experts, the nursing informatics course was already incorporated
into the nursing curriculum during the year 2008 through the Commission on Higher Education(CHED) Memorandum Order 5, Series of 2008. This was later revised and revived as Health
Informatics by CHED Memorandum Order 14 implemented in the summer of 2010 (Sumabat,2010). The current batch of nurses moving into the workforce has already been exposed to thecourse considering the timeframe of implementation. This may have provided the impetus tobe confident in dealing with the e-learning environment.
Being an educator of nursing informatics at the university, the proponent reinforced that thecourse is taught at the nursing academe following the framework set by international experts.
A primary reference of the course was based on the works of Staggers et al. (2001) who
emphasized the competencies of computer skills, informatics competencies, and informatics
skills. Provision of the course in the undergraduate level may have influenced the e-learning
self-efficacy of the participants. Availability and utilization of research-based materials further
molded the development of nursing informatics on a standardized scale.
Higher scores on items dealing with basic computer and online skills reinforced the notion that
nursing informatics in the country is superficially taught covering only basic computer skills(Bernal et al., 2008; Sumabat, 2010). It is also highly unlikely that operating room nurses haveenrolled in post-graduate education, which covers advanced informatics competencies as thisis only highly required in the academic field. Another perspective may also be given on theexperiential point of view as the scope of nursing in the area includes the application of skills

63

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019such as electronic encoding of surgery-related documents, use of word processing documents,
accessing the internet and file downloading and uploading. Results obtained by the respondents
on Cluster A (Basic Computer Skills) and Cluster C (Basic Online Skills) indicators supported
this claim.
The indicators relating to Cluster B (Complex Computer Skills) is an area of concern. The data
supports the perspective of local experts that only basic computer skills are focused on in thenursing informatics education of students in the Philippines (Bernal et al., 2008; Sumabat,2010). The framework made by Staggers et al. (2001) regarding informatics competencies at 4
levels of practice also suggests that these skills belong to a higher level of complexity. Thisline of logic and the traditional delivery of education in the country also supports the need theimprove Cluster F (Utilization of E-learning Platform) indicators. As of date, there are only ahandful of academic institutions in the country fully equipped and utilizing online platforms.
Cluster D (Online Etiquette) was also pointed out as an area of concern. It is quite surprisingwith the participants rating themselves good with basic online skills; there still remains a needto improve online etiquette. Online etiquette involves behaviors over the net that portrays
respect to others in the internet community. The course content of nursing informatics course
offered locally was reviewed, and it was found out that there is no specified unit to tackle theconcept of online etiquette, but this might be discussed along with the topic of online privacy
and security.
Cluster E (Troubleshooting) indicators apparently are problematic. Nurses are not trained tosolve computer-related issues by profession. In fact, the informatics competencies collectively
merged by Staggers et al. (2001) along the four levels of practice, and the list of competencies
presented by the Technology Informatics Guiding Educational Reform (TIGER) (Fung, 2016)
did not include troubleshooting of computer hardware, software or networking as an essential
skill of a nurse.
Limitations
The exposure of participants to e-learning and similar technology along their course of work
may affect the findings of this study. As of date, Cebu Doctors’ University Hospital (CDUH)
has started incorporating an electronic health record and hospital information system in
selected units of the institution such as the emergency room and pharmacy. These systems have
yet to be streamlined into the overall operations of the hospital. The participants may have beenoriented to these applications according to their previous assignments and rotation. Some staff
nurses may also have enrolled in post graduate programs which deliver courses through elearning and other online modalities.
Recommendations
This paper has dealt with e-learning self-efficacy as a major indicator of e-learning readiness.
It is one of the major prerequisites to initiate and sustain e-learning as a new modality ofeducation and training among operating room nurses. In line with this, it is recommended to
explore and look into other domains of readiness to fully utilize and harness the potential ofthis technology into the field of health care. Literature offers other factors that may influencee-learning readiness such as availability of human resources, access to technology,
infrastructure support and personal attributes including study habits, motivation, time
management and self-directed learning. With the findings that the operating room nurses are64

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

in the positive spectrum of readiness in terms of e-learning self-efficacy, future researches may
be conducted to develop, incorporate and determine the effects of e-learning towards theeducation and training of these care providers.
Conclusion
The operating room nurses of the selected hospital have positive e-learning self-efficacy basedon the item analysis and secondary indicator-clusters analysis. Item analysis reflected the
distribution of responses and item means of the respondents to be on the higher end level ofself-efficacy based on the Modified E-learning Readiness Assessment Tool (MERAT). Cluster
analysis also revealed that the participants perceived themselves to be “Very Good” in termsof complex computer skills, online etiquette, troubleshooting computer-related issues andusage of an e-learning platform, and “Excellent” in terms of basic computer and online skills.
The operating room nurses involved in the study portray individuals with good self-efficacy(Bandura, 1997) as those who recognize the challenges and seek the full potential of online
learning. Hence, e-learning initiatives may still diffuse well into the health care system toaugment education and training of nurses despite manpower shortage and issues surrounding
the incorporation of informatics competencies into their knowledge and skills base.

65

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
Akaslan, D., & Law, E. L.-C. (2011). Measuring teachers’ readiness for e-learning in higher
education institutions associated with the subject of electricity in Turkey. 2011 IEEE
Global Engineering Education Conference (EDUCON).
https://doi.org/10.1109/educon.2011.5773180
Alqurashi, E. (2016). Self-efficacy in online learning environments: A literature review.
Contemporary Issues in Education Research (CIER), 9(1), 45–52.
https://doi.org/10.19030/cier.v9i1.9549
Arimbuyutan, R. C., Kim, S., Song, J., & So, W. (2007). A study on e-learning for
Philippines. International Journal of Multimedia and Ubiquitous Engineering, 2(4),49–54. https://doi.org/10.14257/ijmue.2007.2.4.05
Aydin, C. H., & Tasci, D. (2005). Measuring readiness for e-learning: Reflections from anemerging country. Educational Technology & Society, 8(4), 244–257.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman.
Bernal, A. B. S., Tolentino, P. A. P., Gavino, A. I., Fontelo, P., & Marcelo, A. B. (2008).
Nursing informatics: Challenges to Philippine nursing curriculum. AMIA Annual
Symposium Proceedings. Nov. 6, 876.
Demir, Ö., & Yurdugül, H. (2015). The exploration of models regarding e-learning readiness:
Reference model suggestions. International Journal of Progressive Education, 11(1),173–194. https://doi.org/10.18844/ijire.v1i1.117
Dray, B. J., Lowenthal, P. R., Miszkiewicz, M. J., Ruiz‐Primo, M. A., & Marczynski, K.
(2011). Developing an instrument to assess student readiness for online learning: avalidation study. Distance Education, 32(1), 29–47.
https://doi.org/10.1080/01587919.2011.565496
Doculan, J. A. D. (2016). E-Learning readiness assessment tool for Philippine higher
education institutions. International Journal on Integrating Technology in
Education, 5(2), 33–43. https://doi.org/10.5121/ijite.2016.5203
Fung, K. Y. M. (2016). Utilizing TIGER competencies to improve informatics practice.
Doctor of Nursing Practice Projects,76. School of Nursing and Health Professions:
University of San Francisco.
High Level Forum on the Health MDGs (2004) Summary of discussions and agreed action
points. Geneva: High Level Forum. http://www.who.int/hdp/en/summary.pdf
Hung, M.-L., Chou, C., Chen, C.-H., & Own, Z.-Y. (2010). Learner readiness for online
learning: Scale development and student perceptions. Computers & Education, 55(3),1080–1090. https://doi.org/10.1016/j.compedu.2010.05.004
Institute of Health Policy and Development Studies. (2006). Migration of health workers:
Country case study Philippines. Geneva: International Labour Office.
http://www.ilo.org/public/english/dialogue/sector/papers/health/wp236.pdf
International Council of Nurses. (2006). The global nursing shortage: Priority areas forintervention. Geneva: International Council of Nurses.
Jan, S. K. (2015). The relationships between academic self-efficacy, computer self-efficacy,
prior experience, and satisfaction with online learning. American Journal of Distance
Education, 29(1), 30–40. https://doi.org/10.1080/08923647.2015.994366

66

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Kadivar, M., Seyedfatemi, N., Zolfaghari, M., Mehran, A., & Hossinzade, Z. (2016). The
impact of virtual-based education on nurses’ self-efficacy in the level II neonatalcare. Journal of Critical Care Nursing, 9(4). https://doi.org/10.17795/ccn-9281
Kuo, Y.-C., Walker, A. E., Schroder, K. E., & Belland, B. R. (2014). Interaction, internet
self-efficacy, and self-regulated learning as predictors of student satisfaction in online
education courses. The Internet and Higher Education, 20, 35–50.
https://doi.org/10.1016/j.iheduc.2013.10.001
K-12 Blueprint (2014). Learning management system (LMS) guide.
https://www.k12blueprint.com/sites/default/files/Learning-Management-SystemGuide.pdf
Lim, C. K. (2001). Computer self‐efficacy, academic self‐concept, and other predictors ofsatisfaction and future participation of adult distance learners. American Journal ofDistance Education, 15(2), 41–51. https://doi.org/10.1080/08923640109527083
Lin, Y.-C., Liang, J.-C., Yang, C.-J., & Tsai, C.-C. (2013). Exploring middle-aged and older
adults’ sources of internet self-efficacy: A case study. Computers in Human
Behavior, 29(6), 2733–2743. https://doi.org/10.1016/j.chb.2013.07.017
Littlejohn, L., Campbell, J., Collins-McNeil, J., & Thembisile, K. (2012). Nursing shortage:
A comparative analysis. International Journal of Nursing, 1(1), 22–27.
https://doi.org/10.1.1.300.9995
Maertens, H., Madani, A., Landry, T., Vermassen, F., Herzeele, I. V., & Aggarwal, R.
(2016). Systematic review of e-learning for surgical training. British Journal ofSurgery, 103(11), 1428–1437. https://doi.org/10.1002/bjs.10236
Moftakhari, M.M. (2013). Evaluating e-learning readiness of faculty of letters of Hacettepe.
Ankara, Turkey: Hacettepe University.
Momanyi, N. B., & Kaimenyi, C. K. (2015). An investigation into factors causing high nurse
turnover in mission hospitals in Kenya: A case for Pcea Chogoria
hospital. International Journal of Academic Research in Business and Social
Sciences, 5(3). https://doi.org/10.6007/ijarbss/v5-i3/1499
Oliver, R. G. (2001). Assuring the quality of online learning in Australian higher education.
Proceedings of 2000 Moving Online Conference (pp.222-231). Gold Coast, QLD.:
Norsearch Reprographics
Pellas, N. (2014). The influence of computer self-efficacy, metacognitive self-regulation andself-esteem on student engagement in online learning programs: Evidence from thevirtual world of Second Life. Computers in Human Behavior, 35, 157–170.
https://doi.org/10.1016/j.chb.2014.02.048
Simmering, M. J., Posey, C., & Piccoli, G. (2009). Computer self-efficacy and motivation tolearn in a self-directed online course. Decision Sciences Journal of Innovative
Education, 7(1), 99–121. https://doi.org/10.1111/j.1540-4609.2008.00207.x
Smith, P.J. (2005). Learning preferences and readiness for online learning. Educational
Psychology: An International Journal of Experimental Educational Psychology,
25(1), 3–12.
Staggers, N., Gassert, C. A., & Curran, C. (2001). Informatics competencies for nurses atfour levels of practice. Journal of Nursing Education, 40(7), 303–316.

67

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Sumabat, K. R. (2010). History of nursing informatics in the Philippines.
http://cebunursinginformatics.weebly.com/uploads/5/0/0/6/5006028/history_of_nursi
ng_informatics.pdf
Takase, M. (2010). A concept analysis of turnover intention: Implications for nursing
management. Collegian, 17(1), 3–12. https://doi.org/10.1016/j.colegn.2009.05.001
Tang, Y., & Tseng, H. W. (2013). Distance learners’ self-efficacy and information literacy
skills. Journal of Academic Librarianship, 39(6), 517–521.
https://doi.org/10.1016/j.acalib.2013.08.008
Tubaishat, A., & Lansari, A. (2011). Are students ready to adopt e-learning? A preliminary ereadiness study of a university in the Gulf Region. International Journal of
Information and Communication Technology Research, 1(5), 210–215.
Valtonen, T., Kukkonen, J., Dillon, P., & Väisänen, P. (2009). Finnish high school students’readiness to adopt online learning: Questioning the assumptions. Computers &Education, 53(3), 742–748. https://doi.org/10.1016/j.compedu.2009.04.014
Watkins, R., Leigh, D., & Triner, D. (2008). Assessing readiness for e-learning. Performance
Improvement Quarterly, 17(4), 66–79. https://doi.org/10.1111/j.19378327.2004.tb00321.x
Womble, J. C. (2007). E-learning: The relationship among learner satisfaction, self-efficacy,
and usefulness. (Unpublished doctoral dissertation), Alliant International University,
San Diego.
Corresponding author: Paul John G. Aventurado
Contact email: pjaventurado0521@gmail.com

68

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

eHealth Literacy of High School Students in the Philippines
Mark Kenneth S. Camiling
University of the Philippines, Diliman, Philippines

69

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
The Internet has become a chief health information source. It is posited that people who use
the Web for health information must possess requisite eHealth literacy to avoid misuse. eHealthliteracy refers to the skills in searching, retrieving, understanding, evaluating, and applying
online health information to alter or maintain well-being. This research employed mixed
methods of data collection, following a three-phase explanatory sequential design: 1) survey
and eHealth literacy test, 2) focus group discussion, and 3) observational study, to describe thestate of eHealth literacy of high school students in terms of their online health information
seeking behaviors, search, retrieval, and evaluation strategies, actual eHealth literacy levels,and perceived eHealth literacy levels. Furthermore, the research aimed at investigating therelationships among school type, actual eHealth literacy levels, and perceived eHealth literacy
levels. Results show that students are adept in searching and retrieving online healthinformation. However, their evaluation strategies are suboptimal as they lack ability to checkfor content accuracy and source credibility. Findings also reveal a significant relationshipbetween school type and perceived eHealth literacy levels, X² (2, N=263) =14.21, p=0.00,
α=0.05. Private school students tend to score their perceived eHealth literacy levels higher.
There is also a significant difference between actual eHealth literacy levels and perceived
eHealth literacy levels, with students rating their perceived eHealth literacy levels (M=32.45,
SD=6.10, N=40) higher than their actual eHealth literacy levels (M=29.13, SD=2.92, N=40),t(40)=4.16, p=0.000, α=0.05. It is concluded that while students are adept in computer andtraditional literacies, they still need to further develop health and information literacies toeffectively evaluate and apply the searched and retrieved online health information.
Keywords: eHealth, eHealth literacy, high school students

70

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

According to Internet Live Statistics (2016), about 50% of the world population is connectedto the Internet. In the local setting, the Philippines registered an Internet Penetration Rate (IPR)of 43.5%. Furthermore, according to Cruz (as cited in Bristol, Caro, Mangaliman, & Bernarte,2016), 60% of Filipino Internet users belong to the youth category of 15-19 years old, thelargest representation in national figures. In the report of Bayani (2014, para. 8), global
information and measurement company, Nielsen, found that a few of the top online activities
of Filipinos are logging on to social networking sites, participating in online discussions andchatrooms, and conducting academic research.
“The Internet, with its capacity to provide information that transcends time and space barriers,
continues to transform how people find and apply information to their own lives” (Seckin,
Yeatts, Hughes, Hudson, & Bell, 2016, p. e161). Hence, the emergence of the Internet has
rapidly changed the paradigm of information consumerism, particularly in the health sector.
Fox and Duggan (2013) found that 72% of Internet users in the United States searched foronline health information. Tracing the early years of the 20th century, one in three adolescentsused the Internet as a reference for health information (Lenhart, Madden, & Hitlin, 2005). Fox
and Duggan (2013) further found that the majority of Internet users sought information about
a specific condition or disease to medically diagnose themselves. Moreover, 28% of these users
did not follow up further with a medical professional. According to Atienza, Hesse, Gustafson,
and Croyle (as cited in Seckin et al., 2016), they generally consult the Internet first before their
physician. These reports imply a strong, apparent influence of the Internet in the way people
manage their health and well-being.
Though access to online health information may develop the consumer to become empowered
and reflexive, Im (as cited in Seckin et al., 2016) strongly declares that regardless of one’s levelof reflexivity and empowerment, a person must still possess a requisite high level of healthliteracy to avoid information misuse. Health literacy has blended in and evolved with theadvancement of technology and onset of Internet access. Hence the emerging concept ofeHealth literacy is coined and explored.
eHealth literacy is defined as “the ability to read, use computers, search for information,
understand health information, and put it into context” (Norman & Skinner, 2006a, p. e27).
Simple as it may seem, eHealth literacy is a multi-skill competency. The ability to navigate thecomputer and access health information requires complex processes. Understanding healthknowledge involves critical evaluation of the acquired information. Lastly, putting theinformation into real-life context entails careful reflection while being faced with various
decision-making mediators.
There have been numerous studies that have attempted to provide descriptions of, indicatorsof, and/or baseline data on eHealth literacy of various populations such as adolescents(Ghaddar, Valerio, Garcia, & Hansen, 2012), adults (Marazienè, Klumbienè, Tomkevičiūtè, &Misevičienè, 2012), and health information and services consumers in general (AlGhamdi &Moussa, 2012). However, these studies were conducted by foreign researchers. Hence, there isa wide gap between the global research development on the emerging concept of eHealthliteracy and the findings from within the Philippines.
Apart from the studies previously mentioned, it is imperative to initiate research on thisemerging field because the advent of technology has prompted schools to revitalize theircurricula in order to respond to the needs of the learners. To contextualize, competencies on
digital literacy and consumer health have been incorporated into the Enhanced K-12 Basic
Education Curriculum of the Philippines since its implementation in 2012. Though there is no71

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

competency specifically targeting the development of eHealth literacy, its components aretaught separately across subjects and grade levels.
The rationale for this research is the need for immediacy to address the absence or lack ofcontextualized studies on eHealth literacy. This research aimed to consolidate the theoreticalunderpinnings of the concept in an attempt to quantify the eHealth literacy of the most ideal
sample, adolescents, particularly those who are transitioning from secondary to tertiary
education, where the level of independence and reflexivity is increased. This study focuses on
Filipino high school students and specifically investigated the following research questions:
1. What is the state of eHealth literacy of high school students in the Philippines in termsof online health information-seeking behavior, strategies for search and retrieval of
online health information, criteria for evaluating online health information, andperceived and actual eHealth literacy levels?
2. What relationship exists among school type, perceived eHealth literacy level, and actual
eHealth literacy level?
3. Is there a significant difference between the perceived and actual eHealth literacy levelsof high school students in the Philippines?
Literature Review
Defining eHealth Literacy
eHealth literacy is a combination of different literacies simultaneously applied in eHealthconditions such as facing online health information and availing of Internet health services.
Norman and Skinner (2006b) developed the Lily Model (Figure 1) to illustrate the facets ofvarious literacies incorporated in eHealth literacy: traditional literacy and numeracy, andcomputer, media, science, information, and health literacies. The researchers explain that theseare further categorized into two central types or models – analytic and content-specific.
Analytic literacies are a set of skills that are practiced regardless of the content or situation.
Content-specific literacies apply to particular situations.

Figure 1: The eHealth Literacy Lily Model (Norman & Skinner, 2006b,
eHealth Literacy Model, para. 1)

72

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

The lily metaphor used by the researchers seamlessly represents how eHealth literacy operates
in the minds of individuals. In its practice, each of the literacy components is applied
independently, whether simultaneously or consecutively. However, efficacy in the applicationof eHealth literacy greatly relies on the interdependence of all its components. The eHealthliteracy as the pistil, as used by Norman and Skinner (2006b), encapsulates thisinterrelationship – the central concept holding its components [petals] which have theirrespective characteristics.
Online Health Information Search, Retrieval, and Evaluation
Online health information-seeking behaviors, patterns, and skills have become an emerging
field in health research over the past two decades. For example, in the study of Eysenbach andKohler (2002), it was uncovered that people find it “easier to access the quality of informationon the [Inter]net than elsewhere because they could verify and cross check the information ondifferent sites” (p. 575). However, it also discovered that their search techniques of the online
users are suboptimal. It means that they are not utilizing effective strategies to retrieve accurate,complete, and reliable information. The researchers further reported that this state is due toreasons such as inability to utilize medical portals or sites of medical societies or libraries as astarting point and usually choosing the first results displayed by the search engine without
exploring the succeeding pages. The participants were also asked about what they consider inchecking the website reliability. They mentioned criteria such as professional layout andwriting, indication of official authority, and citation of scientific references.
Measuring eHealth Literacy
Little is known about how to quantitatively measure eHealth literacy. Pleasant (2009) argues
that while existing tests are comprehensive, they are merely screening tools. He explains that“the goal of screening is to divide people into healthy and sick categories and it does not tellwhat is actually wrong with the patient” (p. 18). In the same light, Baker (2006) posits that themeasurement of literacy must also involve the individual’s reading ability and vocabulary, notonly their health status. Considering that literacy operates through interactive relationshipamong individuals and the healthcare system, he further stressed that measuring thecommunication capacities of the person is also necessary.
As the instruments evolve to cater to the changing profiles of the population, contexts at which
the individuals are assessed adjust as well. According to the Institute of Medicine (as cited inNorman & Skinner, 2006a), there is “a need to look at the different contexts where healthinformation is obtained and used as part of a strategy of addressing health literacy”(Introduction section, para. 4). These contexts include electronic sources. This has been theinspiration of Norman and Skinner (2006a) in developing the eHealth Literacy Scale(eHEALS) which is “a self-report tool that can be administered by a health professional and isbased on an individual’s perception of her or his own skills and knowledge within eachmeasured domain” (p. e27). It is crafted to measure individual’s estimated literacy level, used
to make clinical decisions. Contextualizing the aims of this present study, it may be inferredthat the instrument most appropriate to use is the eHEALS as it is the only instrument thatprovides if not actual – at the very least perceived – a quantitative measure to describe theeHealth literacy profile of an individual.
eHealth Literacy and Adolescents
It is ideal to identify the youth as a focus for eHealth literacy studies for many reasons. First,
adolescents are about to enter college life, which is “a transitional time for taking charge of
their own health and maintaining a healthy lifestyle, which is essential to their academic
achievement and their overall adult health outcomes” (Ma & Latham, 2013, p. 793). Second,73

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

they are highly familiar with technology as evidenced by their high IPR (Cruz, as cited in
Bristol et al., 2016). This is further supported by Norman and Skinner (2006a) when theysampled youth in their research because of the age group’s high level of eHealth use. Third,
adolescents are found to experience difficulty “engaging with eHealth and understanding or
using health information online” (Gray, Klein, Noyce, Sesselberg, & Cantrill, 2005 in Norman& Skinner, 2006a, Introduction, para. 7). In addition, Ivanitskaya, Boyle and Casey (2006)
concluded in their study that adolescents do not possess the ability to “conduct advanced
information searches, judge the trustworthiness of health-related websites and articles, anddifferentiate between various information sources” (Conclusions, para. 1). The authors further
stated that even if adolescents reported high confidence in perceived eHealth literacy, theirskills are still not an accurate predictor of their health information competencies. It ishypothesized by Norman and Skinner (2006a) that the longer the exposure of an individual totechnology, the more likely s/he to use it as a tool to aid in health management.
Therefore, as adolescents grow in technology-rich environments, they will continue to seek
health information from these digital sources. With this inference, it is of high importance that
continuous research be conducted in order to appropriately and adequately address theadolescents’ eHealth literacy needs through intervention provisions.
eHealth Literacy in the Philippine National Curriculum
As of this writing, there is no competency in the Philippine national health education
curriculum that deliberately and specifically covers the development of eHealth literacy acrosselementary and high school. However, the components of eHealth literacy are taken
independently. For example, sources of reliable information are initially discussed in Grade 3.
It is the terminal lesson in the 3rd Quarter wherein at the end, the students are expected to“[identify] reliable sources of health information” (Department of Education, 2016, p. 36). The
following are the sources of health information explicitly mentioned in the document: a)government agencies, b) health professionals, and c) printed materials produced by healthprofessionals or legitimate health institutions. On the other hand, it is in the health curriculumof Grade 10 when evaluation of health information is tackled. The learning competencies
required of the students after they study consumer health in the 1st Quarter (p.87) are:differentiates reliable from unreliable health information, products, and services; explains theguidelines and criteria in the selection and evaluation of health information, products, andservices; explains the nature and dangers of quackery; and reports fraudulent health services.
Although these topics are not “online” in nature, it must be posited that high school students
are equipped with skills needed in the practice of eHealth literacy.
Methodology
Research Design and Methods
The study employed a three-phase explanatory sequential mixed methods design (Cresswell &Plano Clark, 2007): survey and eHealth Literacy Test (eHLT), focus group discussion (FGD),
and observational study (OS). The first phase aimed at eliciting the respondents’ demographic
profile and online health information-seeking behavior. It was followed by FGD to gather
qualitative data on factors that influence the state of eHealth literacy of the high school students
and their retrieval and evaluation strategies. Finally, the last phase elicited observations on the
personal search and retrieval processes of the high school students given a specific task.
The research was conducted within the school year when it commenced. Furthermore, the data
gathering procedures were carried out consecutively with short intervals. This is due to factors
such as time, experience, exposure to technology, and learning of new concept and skills in74

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

regular class that may affect the perceived and actual levels of eHealth literacy of the highschool students.
Sample and Sampling Procedures
The research was conducted in Quezon City, Metro Manila to ensure variation in student
characteristics. It has also the highest student population in the country. In addition, Metro
Manila registered the highest IPR of 19% in the country (Social Weather Stations, 2013; ascited in Labucay, 2014).
Grade 10 students from four private and four public schools were purposively chosen because
at the time this study has been conducted, they have completed the core competencies of thenational health curriculum, particularly on consumer health.
There were two sampling procedures employed. The first determined the survey and eHLT
participants. The eight schools were chosen through criterion sampling. Medium and large
schools, as categorized by the Department of Education, were considered to ensure maximum
variation. Excluding homogenous classes, the section in each school was selected throughrandom sampling. The number of respondents in each school varied as intact class sampling
was employed. Tables 1 and 2 describe the demographic information of the participants.
Table 1: Population of schools
SchoolSchoolPopulation*

Grade 10
Population

SampleSurveyed

Invalidated
Surveys**

ActualSample
Size

A
B
C
D
E
F
G
H
Total

368
975
297
***
2748
3314
3126
1978

71
168
92
224
622
607
658
510

29
43
39
29
39
40
36
48
303

3
4
3
2
7
4
4
2
29

26
39
36
27
32
36
32
46
274

*both Junior and Senior High Schools**incomplete and incorrectly answered***did not disclose

75

ActualSampleSize by
SchoolType
Private= 128

Public =146

274

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 2: Demographic information of survey and eHLT respondents
SchoolSex:
Male
Female
Age:
15
16
17
18
19
20

A

B

C

D

E

F

G

H

Total

16
10

14
25

22
14

14
13

14
18

13
23

14
18

21
25

128
146

13
11
2
-

11
26
1
1
-

14
16
5
1
-

9
14
3
1
-

15
13
2
1
1

8
20
6
1
1
-

4
16
9
3
-

17
20
7
2
-

91
136
35
7
4
1

The second sampling procedure was carried out to identify the FGD and OS participants.
Through purposive sampling, four schools (a pair from each school type) with the highest andlowest means of perceived eHealth literacy level were considered. Table 3 shows the results ofthe eHLT. Ten students from Schools A, B, E, and H participated in the FGD and OS. Thesestudents were composed of the five respondents who scored the lowest and five who scoredthe highest perceived eHealth literacy levels. Table 4 details their demographic information.
Table 3: eHLT results
Type
SchoolPrivateA
B
CD
Public
E
F
G
H

Mean
31.77
35.21
33.00
32.67
28.44
29.31
30.06
32.47

Table 4: Demographic information of FGD and OS participants
SchoolSex:
Male
Female
Age:
15
16
17
18

A

B

E

H

Total

6
4

2
8

6
4

5
5

19
21

5
4
1
-

1
8
1

4
4
1
1

5
2
3
-

15
18
5
2

76

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Research Instruments
The research instruments utilized in the study are the survey, eHLT, and eHealth Literacy
Rubric (eHLR), which are supplemented with guides for the conduct of the FGD and OS.
a. Survey
The survey is based on the instrument developed by TNS Political & Social (2014) as
published in their research on digital health literacy of European citizens. The first of
two parts collected demographic information. The second part is composed of openand close-ended questions that elicited information on the students’ online healthinformation-seeking behaviors.
b. eHLT
Administered simultaneously with the survey, the eHLT was utilized to investigate therespondents’ perceived eHealth literacy level. It was developed by the researcher as a
derivation of Norman and Skinner’s eHEALS (2006a). It measured the respondents’
perceived level of effectiveness in the completion of tasks related to the application ofeHealth literacy. The respondents assigned a perceived level of effectiveness inaccomplishing each task. The ratings and their corresponding descriptions are listed inTable 5.
Table 5: eHLT rating of task effectivenessRating4
3
2
1

Description
I can do the task alone confidently and effectively.
I can do the task effectively but with minimal assistance from anotherperson.
I can do the task when fully assisted by another person.
I cannot do the task alone or even with assistance from another person.

In the context of online health information search and retrieval to address a healthconcern or problem, Table 6 presents the tasks in the sequence of accomplishment andthe literacies being practiced.
Table 6: eHLT tasks

1.
2.
3.
4.
5.
6.

Task
Understand the information needed to search
for when asked to address a health concern
or problem
Tell whether the technological tool isconnected to the Internet or not
Identify appropriate software applications to
use to search for online health information
Navigate the software application used tosearch for online health information
Identify websites that are credible sources ofhealth information
Check if an unfamiliar website is a credible
source of health information

77

Literacy/ies
Traditional literacy andnumeracy
ComputerComputerComputer
Health and information
Health and information

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

7. Understand the language used in the Traditional literacy andinformation presented
numeracy
8. Identify helpful information from a text Health, information, media,
presented
and science literacies
9. Tell whether the online health information is Health, information, andaccurate or notscience10. Explain the answer to a health concern after Traditional literacy andsearching and retrieving online health numeracy
informationc. eHLRThe eHLR is a researcher-developed rubric used to score the respondents’ actual
performance of eHLT tasks during the simulation activity in the OS.
Prior to data gathering, an expert validation and a pilot study were administered to ensure thevalidity and reliability of the research instruments. Four content professionals and one language
expert participated in the expert validation. The pilot study was conducted in two secondary
schools: one public and one private.
Having an expert validation rating of Exceeds expectation (Survey: M=3.75/4.00; eHLT:
M=3.91/4.00; eHLR: M=3.81/4.00), Good Cronbach’s alpha (α) of 0.82 from the pilot study,
and no major recommendations for language revision, the results of the instrument validationand pilot study indicated that the research tools are valid and reliable.
Data Analysis
The resulting data from the various methods of collection were rendered and analyzed throughdifferent means. The data analysis procedure is subdivided into three phases.
Phase one. Descriptive statistics were employed to organize survey data which include thedemographic profile and the prevalence and patterns of online health information-seeking
behaviors.
The eHLT was scored accordingly, with minimum score of 10 and maximum of 40. This
determined the perceived eHealth literacy level. As the score adds, the level increases. Table 7
indicates the range of scores and corresponding proficiency levels. This is based on the studyof Cutilli and Benett (2009, Results of the NAAL section), where adult health literacy in the
United States was evaluated through a series of tasks. Proficient eHealth literacy refers to theability to perform complex tasks such as knowing how to properly use online healthinformation and evaluate content validity and source credibility. Intermediate eHealth literacy
refers to skills that allow the person to perform moderately complex tasks such as knowing
where to find helpful health information on the Internet. Basic eHealth literacy refers to theability to perform simple tasks such as accessing the Internet. Below basic eHealth literacy
describes the skill level below simple and concrete tasks, which include the inability to use
technology to access online health information.

78

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 7: eHLT proficiency levels
Score
29 – 40
24 – 28
19 – 23
10 – 18

Level
ProficientIntermediate
Basic
Below Basic

Phase two. The results of the FGD were rendered through thematic analysis. On the other hand,each respondent’s effectiveness in the performance of eHealth literacy-related tasks during the
OS was scored accordingly using the eHLR. The same scoring and assignment of literacy levels
used in the eHLT evaluation were utilized. The results from the eHLR yielded eachparticipant’s actual eHealth literacy level.
Phase three. Using IBM SPSS Statistics version 20 for Windows, inferential statistics were
employed to answer Research Questions 2 and 3. A chi-square test was administered todescribe the relationship that exists among school type, perceived eHealth literacy level, andactual eHealth literacy level. In addition, a paired samples t-test was administered to determine
if there is a significant difference between the means of perceived and actual eHealth literacy
levels of the high school students.
Ethical Considerations
Several ethical guidelines were upheld in the conduct of the study. The well-being of the
respondents was protected at all times, with their identities kept confidential. Necessary permits
were also secured from appropriate channels prior to administration of any data gathering
procedure. Informed consent, verbal assent, and written consent were also secured whereverimperative to do so. No individual was forced to participate in the study and all were given
freedom to leave the study any time they feel the need to do so.
Results and Discussion
Access to the Internet and Online Health Information
It is not surprising that 100% of the respondents in this study reported access to the Internet,
with personal gadgets used most frequently (at 92.0%). The age group of the respondents
makes up the biggest populace of Filipino Internet users. According to On Device Research(2014), 53% of Filipino Internet users are under the age of 24. In terms of frequency of use,
89.8% of the respondents reported daily access to the Internet. This percentage even surpasses
the finding of Labucay (2014) that three-fifths of Filipino Internet users are “frequent tomoderate users” (p. 39), accessing the Web a few times or at least once daily.
Of those who access the Internet, 96% reported searching for online health information, withpersonal gadgets remaining to be the top choice for use (at 89.4%). Of this percentage, 47.5%
conveyed that they do so weekly, while 41.4% reported monthly access. The high percentageof reported access to online health information even exceeds the range of 40%-73.4% describedin existing literature on adolescents (Jimenez-Pernett, de Labry-Lima, Bermudez-Tamayo,
Garcia-Gutierrez, & Salcedo-Sanchez, 2010). These also support the findings of Labucay(2014) as she reported that health, dieting, and fitness information-seeking and -learning are
one of the major online activities of Internet users.

79

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Type of Health Information Searched Online
The types of health information searched for by the respondents vary. In general, most of theinformation gathered by the students is about personal health (60.8%), nutrition (58.2%), andinjury prevention, safety, and first aid (52.5%). This finding supported the study of FernandoCallo (2009) on high school students’ expressed health interests. She reported that the top threehealth interest topics are about mental health, nutrition, and safety education and first aid.
These concerns are linked to health risk behaviors reported by Peltzer and Pengpid (2015) intheir cross-sectional study on the results of the 2003, 2007, and 2011 Global School-based
Health Survey to investigate the trends in the health risk behaviors of adolescents aged 10-19.
These behaviors include improper dieting, lack of physical activity, increased sedentary habits,
injury and violence involvement, and practices that contribute to poor mental health. According
to Miguel-Baquilod (as cited in Peltzer & Pengpid, 2015, p. 1), risk behaviors relative to theaforementioned list are correlated with the leading causes of death among the youth in thePhilippines.
Peltzer and Pengpid (2015) reported that there were significant improvements in malnutrition
and mental health over the nine-year coverage of the study. However, it has been found that
there is an increase of health risk behaviors that pertain to bullying, injury, and loneliness. Thisparticular finding of their research is indicative of this study’s results on the specific healthconcerns being searched for online by the respondents.
The high percentage of information search about mental health also supports the results of the2015 National Youth Assessment Study reported by Cendeña (2017). Cendeña mentioned that
20% of the 2,762 respondents “have experienced extreme emotional distress” (p. 20), while
about a third “thought at least once that life was not worth living” (p.20). This may be anunderlying reason why this study’s participants reported the highest frequency of online
information search for mental health concerns.
Sources and Consumers of Online Health Information
Results show that 63.5% of the respondents use Google Chrome, Mozilla Firefox, Bing, Yahoo,
Safari, and Internet Explorer for health information search. These bridge the Internet users tovarious websites that offer health content. Boyer & Geissbuhler (2006, in Jimenez-Pernett etal., 2010) posit that these mechanisms are most frequently used “because they are very easy,
accessible, fast, and contain a great deal of information” (p.5).
At 42.6%, the second most frequently visited source of online health information are social
media sites. Official websites of government and non-government health organizations ranked
fourth at 14.4%. Online newspapers and magazines are reported to be the least common sourceof online health information at 12.5%.
Jacobs, Amuta, and Kwon (2017) mentioned that the motivations behind the healthinformation-seeking behavior are the individual’s perception of health, health status, andfamily health history. This is consistent with the results of this study’s survey. When asked forwhom the health information search is for, 72.6% of all respondents reported that they are themere consumers of the information. Only 25.1% searches online health information for theirfamily members and a low of 2.3% for their friends.
The survey also prompted the respondents to identify all possible reasons for online healthinformation search. A high percentage of 65% stated that they wanted to gain additional
knowledge on the topics searched. It is interesting to note that private and public schools differ
80

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

on the second most common reason for searching online health information. Among students
from public schools, 13.0% reported that they seek clarification in the Internet regarding theopinion they gathered from a medical check-up. In the private school setting, 8.0% of thestudents indicated that they search for online health information to guide them in buying ahealth product or availing of a health service.

Number of Students

Perceived eHealth Literacy
The results of the eHLT revealed that the respondents perceive their eHealth literacy level as
proficient at M=31.75 (n=263). The mean eHLT scores of students from private and publicschools are 33.30 (n=125) and 30.30 (n=138) respectively. These results indicate that generally,
the students’ eHealth literacy level is proficient. Figure 2 indicates the complete eHLT results.
120
110
100
90
80
70
60
50
40
30
20
10
0

PublicPrivateBelow Basic

Basic
Intermediate ProficientProficiency Level

Figure 2: Perceived eHealth literacy levels
The eHLT also indicated that the participants are most confident in efficiently performing tasks
that practice their computer and traditional literacies. These include the ability to understandthe information needed to retrieve to address the health concern (M=3.21), to tell whether the
tool is connected to the Internet or not (M=3.35), to identify the software application needed toretrieve online health information (M=3.22), to navigate the software application (M=3.16),
and to understand the language used in the presentation of information (M=3.30). Different
factors may explain this trend. First, it has been established that today’s youth are tech-savvy.
Second, this is supported by the high basic and functional literacy rates of the country. In thePhilippine in Figures 2017 Report of the Philippine Statistics Authority (2017), the basic
literacy rate of Filipinos in 2013 is 96.5% while the functional literacy rate is 90.3%.
On the other hand, they scored their lowest efficiency in tasks that apply health and information
literacies. The majority of the respondents reported difficulty and need of full assistance fromanother person in explaining the answer to a health concern after an information search(M=3.15), identifying credible websites (M=3.10), checking the credibility of an unfamiliar
website (M=3.01), and proving the accuracy of the online health information (M=2.98).

81

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Focus Group Discussion
FGD results revealed that the participants preferred Google Chrome as their starting point insearching and retrieving online health information due to its speed and variety of references.
The participants also reported the use of both English and Filipino in accessing online healthinformation. There are more participants from private schools who type keywords on the searchengine because for them, it will provide more varied scope. The sentence or question form ismore preferred by participants from public schools as they think it provides them exact
information they need to address their concern or question. When asked which from the Google
results list is first chosen, the majority of the respondents reported that they select the firstwebsite because they think it is the most rated and reliable.
Upon analyzing the emerging themes, there are only three main criteria used by the respondents
to evaluate the credibility of the online health information: user rating and review, commonalityof content across websites, and authority of source. It may be deduced that the criteria aresomewhat ineffective. For one, user rating and review may not necessarily guarantee theaccuracy of content. The anonymity of the source is already a cause of doubt and worry.
Moreover, the commonality of content across websites is neither a reliable criterion because
information can easily be disseminated online by anyone. It is only the authority of source thathas high reliability in terms of discerning the trustworthiness of the source. In terms of action
taken after information retrieval, the majority of the respondents reported that they immediately
applied the information to address their concerns through self-medication, which has no
guarantee of effectiveness and may pose great harm.
Observational Study
The search and retrieval process was simulated during the OS. The 40 respondents visited a
total of 113 webpages from 35 unique websites. The average time in finishing the search andretrieval process is 7 minutes 20 seconds. All participants used Google Chrome as a starting
point and Google Search as the engine for information retrieval. With relatively fast durationof search and familiarity with online navigation, it was observed that the students are adept interms of their computer literacy.
The majority of the health concerns searched were phrased in English. In analyzing the qualityof search, it is imperative to validate whether the phrase or sentence searched is accurate to theconcern being addressed. Although the majority of the participants scored high in their ability
to understand the information needed to search (M=3.55), which is a skill practiced in
traditional literacy, there is disparity in terms of phrasing the information. There were instances
when the information typed in the search bar included unnecessary details. In addition, none ofthe participants utilized Boolean operators for a more efficient search.
About 60% of the participants clicked the first website in the results list. 90% of the respondents
reported lack of familiarity with the websites they visited. When asked to check for credibility,
almost half admitted to having no idea. When probed by the researcher, the students mentionedthat they tend to consider the physical layout or attractiveness of the website as the primarybasis to determine its credibility.
Identifying helpful information in the article is not considered a challenge for the respondents.
With the mean score of 3.40, it may be analyzed that the participants’ facility in the English
and Filipino languages aided them in the understanding of information that will answer thehealth concern. However, checking the information’s accuracy seems to be a challenge for theparticipants. A low mean score of 1.43 in the particular task only dictates that the participantscannot verify the information even if assistance and cuing are given. From the abovementioned
82

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

observations, it is inferred that the students need further development of their health,information, media, and scientific literacies.
At the end of the OS, the participants were required to explain to the researcher the information
gathered to address the health concern. With a mean score of 3.05, it is inferred that they are
equipped in terms of comprehension and summarizing skills.
Actual eHealth Literacy
The results of the OS complement the reported perceived inefficiency of the respondents inapplying their health and information skills. The general mean score of students in the OS is29.13 (n=40), which translates to borderline intermediate to proficient eHealth literacy level.
Unpacking the results further, students from private schools scored lower in the search andretrieval experiment (M=28.85, intermediate eHealth literacy) than those from public schools(M=29.40, proficient eHealth literacy). Figure 3 indicates the participants’ actual eHealthliteracy scores.

Number of Students

15129Public6

Private30Below Basic

Basic
Intermediate ProficientProficency Level

Figure 3: Actual eHealth literacy levels
Relationships among Variables Affecting eHealth Literacy
The chi-square test of independence also indicated that the actual eHealth literacy level does
not depend on the school type, X²(2, N=40)=0.99 with p-value of 0.61 at 95% confidence.
However the results of the same test indicate that school type affects the perceived eHealthliteracy level, X²(2, N=263)= 14.21with p-value of 0.00 at 95% confidence. Students fromprivate schools tend to score their perceived eHealth literacy level higher than theircounterparts in public schools.
Lastly, a paired samples t-test was administered to investigate if there is a significant difference
between the perceived and actual eHealth literacy levels of the high school students. Results
show that there is a significant difference between the two variables. The high school students
rated their perceived eHealth literacy (M=32.45, SD=6.10, N=40) higher than their actual
eHealth literacy (M=29.13, SD=2.92, N=40), t(40)=4.16, p=0.000 at 95% confidence.

83

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Summary and Conclusion
This study aimed at describing the state of eHealth literacy of high school students in terms of
their online health information-seeking behavior, strategies for search and retrieval of online
health information, and criteria for evaluating online health information. It also intended toprovide a quantitative description of the perceived and actual eHealth literacy of the students
along with identifying their relationship with school type.
The high school students indicate high frequency of use of personal gadgets to search for online
health information to maintain or improve their well-being. They prefer to use search engines
as a starting point. They generally search for information about personal health, nutrition, andinjury prevention, safety and first aid. Personal consumption was the main motivation for the
online search. When asked for the general satisfaction, almost all participants provided anaffirmative response.
The high school students’ ability to evaluate the credibility of online sources of healthinformation and the accuracy of health information in general is suboptimal. They tend to
consider the layout of the website rather than its content. It may be concluded that the
respondents have not mastered the curricular competencies covering these skills. It is in thenational health curriculum of Grade 10 (p. 87) when evaluation of health information is tackled.
It is essential to point that the participants of the study are Grade 10 students and the research
was conducted during the 4th Quarter; this means that these competencies should have beenlearned by the students already.
The high school students perceive their eHealth literacy level higher than their actual ability.
This finding implies that they may not have been given enough opportunities to practice theirlearning competencies on evaluation of health information and health information sources.
Knowledge may have been imparted, but its actual application may be missing.
In summary, the high school students, though adept in terms of traditional and computer
literacies, still need to further develop their scientific, media, information, and health literacy
in order to effectively consume and apply online health information to maintain and/or improvetheir well-being.
Recommendations
It is recommended to conduct further research to investigate other influencing factors to
eHealth literacy. A study on the impact of educators and other stakeholders in the development
and enhancement of eHealth literacy is imperative to expounding on the emerging skill. It isalso essential to develop and validate eHealth literacy measurement tools in the local language.
Since the findings of the study indicate absence/lack of the skill integration in the curriculum,
reviews by program and curriculum planners must be initiated to incorporate the emerging
concept especially in consumer health learning competencies. It is also necessary to reinforce
consumer health instruction in schools through the educational policy-makers, administrators,
and educators in order to strengthen the foundational skills of the students in terms ofevaluating source credibility and content accuracy of health information gathered whether
online or through other emerging forms of media. In terms of classroom assessment, review
and enhancement of methods to evaluate consumer health skills must be proposed so that actual
practice of such among students shall be evident and successful. If curricular inclusion may not

84

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019be possible, it can be recommended that school administration devises an eHealth literacy
development program or module as supplementary instruction in regular classes.
Extending the utilization of the research results to the healthcare setting, medical personnel
may use the data to enhance patient-professional relationship as the services are being rendered.
Furthermore, the findings may also be key in the improvement of user interface of online sitesthat provide health information, products, and services.

85

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
AlGhamdi, K. M. & Moussa, N. A. (2012, June). Internet use by the public to search for
health-related information. International Journal of Medical Informatics, 81(6), 363–
373. https://doi.org/10.1016/j.ijmedinf.2011.12.004.
Baker, D. W. (2006). The meaning and the measure of health literacy. Journal of General
Internal Medicine, 21(8), 878–883. https://doi.org/10.1111/j.1525-1497.2006.00540.x
Bayani, O. (2014, Jun 25). Internet penetration doubles in PH, owed to mobile boom.
Marketing. http://www.marketing-interactive.com/internet-penetration-doubles-phowed-mobile-boom/
Bristol, N. M., Caro, J, J.D., Mangaliman, C.C., & Bernarte, R.P. (2016, February). The
digital media consumption, dependency and its self-perceived effects on familiar andpeer interpersonal relationships of the Filipino youth. Asia Pacific Journal ofMultidisciplinary Research, 4(1), 91–98.
Cendeña, P. (2017). Making sense of the complicated: Highlights of the 2015 National Youth
Assessment Study [PowerPoint slides]. http://yptoolbox.unescapsdd.org/wpcontent/uploads/2017/08/National-Youth-Assessment-Study.pdf
Cresswell, J. W. & Plano Clark, V. (2007). Designing and conducting mixed methods
research. Thousand Oaks, CA: Sage.
Cutilli, C., & Benett, I. (2009). Understanding the health literacy of America results of the
National Assessment of Adult Literacy. Journal of Orthopaedic Nursing. 28(1): 27–
34. https:// doi.org/10.1097%2F01.NOR.0000345852.22122.d6
Department of Education. (2016). K-12 Health Curriculum Guide.
http://www.deped.gov.ph/sites/default/files/Health%20CG.pdf.
Eysenbach, G. & Kohler, C. (2002). How do consumers search for and appraise healthinformation on the world wide web? Qualitative study using focus groups, usability
tests, and in-depth interviews. BMJ, 321, 573–577.
https://doi.org/10.1136/bmj.324.7337.573
Fernando-Callo, L. (2009). Students’ expressed health interest and parents’ perceptions of
their children’s health needs (Unpublished masters thesis). University of thePhilippines-Diliman, Quezon City.
Fox, S. & Duggan, M. (2013). Health online. Pew internet and American life project.
http://pewinternet.org/Reports/2013/Health-online.aspx
Ghaddar, S. F., Valerio, M. A., Garcia, C. M., & Hansen, L. (2012, January). Adolescenthealth literacy: The importance of credible sources for online health information.
Journal of School Health, 82(1), 28–36. https://doi.org/10.1111/j.17461561.2011.00664.x
Gray, N .J., Klein, J. D., Noyce, P.R., Sesselberg, T. S., & Cantrill, J. A. (2005, April). Healthinformation-seeking behaviour in adolescence: the place of the internet. Social
Science and Medicine, 60(7), 1467–78.
https://doi.org/10.1016/j.socscimed.2004.08.010
Internet Live Stats. (2016, Jul 1). Internet users. http://www.internetlivestats.com/internetusers/
Ivanitskaya, L., O’Boyle, I., & Casey, A. M. (2006). Health information literacy andcompetencies of information age students: Results from the interactive online research86

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

readiness self-assessment. Journal of Medical Internet Research, 8(2), e6.
https://doi.org/10.2196/jmir.8.2.e6
Jimenez-Pernett, J., de Labry-Lima, A., Bermudez-Tamayo, C., Garcia-Gutierrez, J., &Salcedo-Sanchez, M. (2010). Use of the internet as a source of health information by
Spanish adolescents. BMC Medical Informatics and Decision Making, 10(6).
https://doi.org/10.1186/1472-6947-10-6
Jacobs, W., Amuta, A., & Kwon, C. (2017). Health information seeking in the digital age: An
analysis of health information seeking behavior among US adults. Cogent Social
Sciences, 3. http://dx.doi.org/10.1080/23311886.2017.1302785.
Labucay, I. D. (2014). Patterns of internet usage in the Philippines. In J. D. James (ed.), Theinternet and the google age: Prospects and perils (pp. 27–49). Dublin: Researchpublishing.net. https://doi.org/10.14705/rpnet.2014.000176
Lenhart, A., Madden, M., & Hitlin, P. (2005 July 27). Teens and technology: Youth areleading the transition to a fully wired and mobile nation. Washington, D.C.: Pew
Internet & American Life Project.
Ma, J. & Latham, D. (2013). Interacting with health information for self-care: A pilot study
exploring undergraduates’ health information literacy. iConference 2013 Proceedings(pp. 793–796). https://doi.org/10.9776/13390
Marazienè, D., Klumbienè, J., Tomkevičiūtè, J., & Misevičienè, I. (2012). Sources andreasons for seeking health information by Lithuanian adults. Medicina (Kaunas),
48(7), 371–378. https://doi.org/10.3390/medicina48070055
Norman, C. D., & Skinner, H. A. (2006a). eHEALS: The eHealth literacy scale. Journal of
Medical Internet Research, 8(4), e27. https://doi.org/10.2196/jmir.8.4.e27
Norman, C. D., & Skinner, H. A. (2006b, Jun 16). eHealth literacy: Essential skills for
consumer health in a networked world. Journal of Medical Internet Research, 8(2),e9. https://doi.org/10.2196/jmir.8.2.e9
On Device Research. (2014, Jul 8). The Philippines mobile internet crowd: Young, affluent, andgrowing [Blog post]. https://ondeviceresearch.com/blog/philippines-mobile-internet-trends
Peltzer, K. & Pengpid, S. (2016). Health risk behaviour among in-school adolescents in thePhilippines: Trends between 2003, 2007, and 2011, a cross-sectional study.
International Journal of Environmental Research and Public Health, 13(1), 73.
https://dx.doi.org/10.3390%2Fijerph13010073
Philippine Statistics Authority. (2017). Philippines in figures 2017.
http://psa.gov.ph/sites/default/files/PIF_2017.pdf
Pleasant, A. (2009). Health literacy measurement: A brief review and proposal. In L.
Hernandez (Ed.), Measures of health literacy: Workshop summary (pp. 17–22).
Washington, D.C.: National Academy of Sciences.
Seckin, G., Yeatts, D., Hughes, S., Hudson, C., & Bell, V. (2016, Jul 18). Being an informed
consumer of health information and assessment of electronic health literacy in anational sample of Internet users: Validity and reliability of the e-HLS instrument.
Journal of Medical Internet Research, 18(7), e161. https://doi.org/10.2196/jmir.5496
TNS Political & Social. (2014). Flash Eurobarometer 404: European citizens’ digital health literacy.
European Union: Directorate-General for Education. https://doi.org/10.2759/86596
Corresponding author: Mark Kenneth S. CamilingContact email: mark.camiling@gmail.com
87

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

88

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Comparative Study Between Traditional and Peer Assisted Learning of Business
Management Students on Accounting Module
Siu Wo Tarloff Im
City University of Hong Kong, Hong Kong
Pit Ho Patrio Chiu
City University of Hong Kong, Hong Kong
Michelle Ng
City University of Hong Kong, Hong Kong

89

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
Candidates who study business disciplines are trained to be all-round administrative staff tohandle business-related issues. Knowledge of a basic accounting principle for supporting
financial decisions is vital in business operations, and is especially crucial for managerial
workers of a listed company. Indeed, the accounting module is one of the core modules forhigher education students who are studying in business disciplines. With the College Heads’
endorsement when departments joined the Peer-Assisted-Learning using Supplemental
Instruction (PALSI) scheme, the academic results of two accounting modules were collectedanonymously for research purposes.
The subject registration information shows that each cohort had at least 300 enrolments and, at
most, over 800 enrolments. A review of academic results throughout four academic years was
performed, the sample size of the study consisting of a total of 1,815 PALSI students and 5,869
non-PALSI students. This article focuses on the quantitative analysis in comparing theperformance of PALSI students and non-PALSI students by monitoring their academic resultsconsecutively in different cohorts. In this study, a t-test is implemented to evaluate theeffectiveness of the peer-assisted learning scheme by identifying the academic results of eachcohort to ascertain if there is a significant difference between the two groups.
Similar investigation on the effectiveness of the PALSI scheme with different accounting
knowledge background before joining the undergraduate programmes is implemented in thelatter part. The findings of the analysis provide remarkable academic enhancement for students
with PALSI support, and it is concluded that the scheme is worth extending to different subjects
in the business discipline in the future.
Keywords: peer-assisted learning, business, accounting, learning support, deep-learning

90

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Introduction
In the past, Hong Kong Certificate of Education Examination (HKCEE) and Hong Kong
Advanced Level of Education Examination (HKALE) candidates were prescribed to selectscience, business, and arts subjects in a specific combination for fulfilling the entry
requirements of the undergraduate programmes in the previous 3-4-3 academic structure – 3
years junior secondary level; 4 years senior secondary level; 3 years undergraduate level. Even
though such modules were available for students to enroll, only the minority of high school
students had the opportunity to take subjects “Principles of Accounts” in HKCEE or HKALE(Hong Kong Examinations and Assessment Authority, 2018a).
Hong Kong Diploma of Secondary Examination
A new 3-3-4 academic structure includes three junior years plus three senior years in secondary
education as well as preparing candidates to enroll in a four-year higher education journey. It
was first implemented in 2012 to offer the Hong Kong Diploma of Secondary Examination(HKDSE). Therefore, professional subjects are likely to be taught at university level rather thanproviding training with limited time in senior secondary school life. The Hong Kong
Examinations and Assessment Authority offers twenty-four senior secondary subjects, which
are four core subjects and twenty elective subjects. HKDSE candidates are required to selectelective subjects to form a study combination on top of the four core subjects - Chinese
Language, English Language, Mathematics, and Liberal Studies.
After implementing the four-year curriculum in higher education, the constraint of selecting
high school elective subjects was removed. Four core subjects plus one elective subject are
the general university entry requirement for HKDSE candidates. Usually, there are no specificentry requirements for enrolling in programmes in the business discipline. By reviewing theregistration records of the HKDSE examination of the last several years, it has been found thatnearly 70% of the candidates have enrolled in four core subjects plus two elective subjects.
Moreover, the accounting subject was not as popular as other elective subjects, that is, biology,
chemistry, or economics.
Accounting modules enrolments of HKDSE. When HKDSE was officially launched in 2012,
there were two subjects related to the accounting discipline. One of them was classified as a
Category A Senior Secondary Subject – “Business, Accounting and Financial Studies” – andanother one was classified as a Category B Applied Learning subject – “Practical Computerised
Accounting” (Hong Kong Examinations and Assessment Authority, 2018b). By reviewing thedata source of the Hong Kong Examinations and Assessment Authority (HKEAA), thestatistics of the subjects’ enrolments were discovered. Table 1 shows that the number ofcandidates for the HKDSE entries in accounting and accounting-related subjects from 2012 to2016 has been dropping continuously, from 17.7 percent to 15.2 percent. The figures conclude
that the accounting subject was not as popular as other subjects in secondary students’ mindset
after the implementation of the new HKDSE curriculum. Therefore, the number of students
with accounting training before entering university was limited.

91

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 1: HKDSE accounting subject enrolments from 2012 to 2017Year

Business, Accounting
and Financial Studies+2012

Accounting++

HKDSE number ofcandidates

Percentage of
Accounting students

12,41270,282

17.7%

2013

11,48269,750

16.5%

2014

10,37565,270

15.9%

2015

9,288

61,136

15.2%

2016

8,300

56,112

14.8%

2017

7,783

51,192

15.2%

Notes:+

The figures are the number of candidates take the elective part “Accounting Module”
After 2015, Business, Accounting and Financial Studies module was reformed into two subjects “Accounting”
and “Business Management”++

Entrance requirements for bachelor’s degree programmes. HKDSE candidates are
responsible for checking the entrance requirements of each undergraduate programme when
they are applying to study. For example, calculus knowledge is an essential tool for engineering
students in science and engineering disciplines, where such specific pre-requisite formathematics knowledge would be an advantage. However, HKDSE Mathematics extended
parts Module 1 (M1) – “Calculus and Statistics” or Module 2 (M2) – “Algebra and Calculus”
is not defined as one of the compulsory subjects in HKDSE level for enrolling into science orengineering disciplines.
In contrast, admission requirements of studying Bachelor of Business Administration
programmes under the College of Business do not have a pre-requisite for accounting studies
at the HKDSE level. In other words, there is no advantage for the students who have taken
accounting as the HKDSE elective module to apply for the undergraduate programmes in theCollege of Business (City University of Hong Kong, 2018).
Accounting modules in undergraduate programmes at the College of Business. In thebusiness world, accounting is a profession about managing financial accounts, auditing, andcommunicating amongst different stakeholders in financial terms. Accountants are required topossess basic accounting knowledge in designing appropriate business models or solving any
financial difficulties of existing business operations. Thus, national institutes and professional
bodies in various countries recognise such professional qualifications mutually in accounting.
It is common to find that Bachelor of Business Administration (BBA) students in the Collegeof Business need to take the accounting discipline modules. BBA students are required to study
either one or two modules related to accounting in the first two years of their study. As shownin Table 2 below, there are thirteen programmes which require undergraduates to take module
CB2100 – “Introduction to Financial Accounting” in Year One. Ten of these programmes also
require module CB2101 – “Introduction to Managerial Accounting” as one of the core modules
in Year Two. Regardless of whether the students are going to select accounting as the major or
the minor discipline of their undergraduate programme, the curriculum dictates these two
subjects. Due to the high demands on these two accounting modules, both courses are usually92

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

offered to two cohorts for each academic year by the Department of Accountancy to serve alarge number of students from the College of Business.
Table 2: Study requirements for students in College of Business
Undergraduate programme

CB2101 - Introduction to
Managerial Accounting

BSc Computational Finance

CB2100 - Introduction to
Financial Accounting
Y

BBA Global BusinessYY

BBA AccountancyYY

BBA Business Economics

Y

BBA FinanceYY

BBA Global Business Systems ManagementYY

BBA Information ManagementY

Y

BBA ManagementY

Y

BBA Business Analysis

YY

BBA Business Operations ManagementY

Y

BBA MarketingYY

BBA Human Resources ManagementYY

BBA International BusinessY

Although these modules are the introductory level for students enrolled in the businessprogrammes, teaching staff might encounter difficulties in handling Year One candidates withdifferent accounting knowledge during the semester. The objectives of our study are to identify
the effectiveness of implementing the Peer Assisted Learning with Supplemental Instruction(PALSI) scheme to support learning for business modules, and recognise that the scheme isstill favourable to learners with subject knowledge background of the module.
Literature Review
Traditionally, teachers usually use the teacher-centred approach in a classroom or a lecture
theatre to deliver knowledge in the lessons of primary school, high school, or university. In
recent years, there are lots of teaching and learning approaches to strengthen the learning
supports for new undergraduates, such as blended-learning, e-Learning, and flippedclassrooms. Educators started adopting the student-centred learning model, as they believe thatthe deep-learning process will happen through indirect knowledge transfer. However, some ofthese approaches might require teaching staff to redesign the course curriculum, apply newteaching strategies, or even plan alternative activities to implement different teaching andlearning tasks. Eventually, faculty staff in universities are reluctant to attempt such a newteaching approach because the teaching effectiveness might be unremarkable.
In recent years, there have been lots of researchers evaluating their learning performances invarious disciplines, such as medical study (Burgess, McGregor, & Mellis, 2014; HerrmannWerner et al., 2017), nursing study (McLelland, McKenna, & French, 2013; Williams,
Olaussen, & Peterson, 2015; Williams & Reddy, 2016), radiography (Meertens, 2016), biology(Rayner & Papakonstantinou, 2018), geography (West, Jenkins, & Hill, 2017), mathematics(Duah, Croft, & Inglis, 2014), accounting (Marrone & Draganov, 2016; Sudhakar, Tyler, &93

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Wakefield, 2016; Supple, Best, & Pearce, 2016), mechanical engineering (Qonda, 2017) andother challenging subjects of undergraduate programmes. However, not many longitudinalstudies have been conducted on the subjects in business disciplines with Peer Assisted Learning
support.
Studies have been done to investigate the effectiveness of peer-assisted learning, andresearchers conclude that academic mean scores are increased by comparison with groups
without peer-assisted learning support (Riaz, 2014; Bennett, Morris, & Mirza, 2018).
Moreover, Williams and Reddy (2016) state that the academic enhancement of student tutors
is most significant.
In the light of this literature, this paper attempts to extend the study on peer-assisted learning
support from science and engineering subjects to modules in business disciplines, and prove
the academic performance of the PALSI group is enhanced effectively. The statistical t-test
was implemented by comparing the academic mean results of the PALSI and non-PALSI
groups in this longitudinal study. Moreover, this study would like to highlight that the peerassisted learning scheme is not only for helping mid-range students, but also for boosting theperformance of top-tier students.
Methodology
The PALSI course nomination form was sent to the College Heads seeking their endorsement
to follow the PALSI guidelines before each semester started. The colleges and departments
agreed that the academic results of the PALSI courses would be collected anonymously for
research purposes, and such a statement was written on the PALSI course nomination form.
During the orientation sessions, an acknowledgement about performance comparison wasmade to the students who wished to join the PALSI scheme. The anonymous analysis was toinvestigate the effectiveness of joining the PALSI scheme in each module. At the end of theacademic year, all these findings were to be compiled into a PALSI report and sent back to thecorresponding departments for reviewing the scheme on different modules.
Statistical tables and charts were compiled to present the academic performance mean value
descriptively with different cohorts. After that, a t-test would be implemented to evaluate theeffectiveness of peer-assisted learning between the PALSI and non-PALSI students. Interestingfindings on the academic performance were revealed by comparing the Grade Point Average(GPA) mean value of PALSI and non-PALSI students. A similar study on the academic results
related to the PALSI scheme on undergraduate mathematics modules has been done before(Im, Chiu, Shek, & Liu, 2017). The previous performance study was also divided into twogroups based on whether they had advanced mathematics training or without advanced
mathematics training before the undergraduate study.
Data Analysis
With collecting the academic results of four academic years, the sample size of the PALSI
students and non-PALSI students are 1,815 and 5,869, respectively. Also, for each cohort, there
were at least 300 enrolments to over 800 enrolments. To study the learning effectiveness ofstudents who have joined the PALSI scheme, we list the grade distribution summary to obtainan overview of the academic performance. In Table 3 referring to CB2100, it shows that the
number of PALSI students in “A range” is nearly the same as the number of non-PALSI

94

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019students in “A range” in most of the cohorts. For 2015-16 Semester B, it finds that over half of
the “A Range” students have joined PALSI.
In another module CB2101, Table 4 shows that PALSI students are not the majority of the “Arange” students; only a few cohorts of PALSI students are dominant in the “A Range”.
However, similar findings on the relationship between the percentage of “A Range” students
and the percentage of PALSI students conclude that they are correlated positively. Two charts– Figure 1 and Figure 2 are created to visualize the relationship of the “A Range” students andthe PALSI students for different cohorts of CB2100 – Introduction to Financial Accounting
and CB2101 – Introduction to Managerial Accounting.
Table 3: Grades distribution for eight cohorts of CB2100
GradesYear/Cohort

PALSI

A+

A

A-

B+

B

B-

C+

C

C-

D

F

2014-15

Y

5

11

15

19

19

17

14

7

5

3

20

Semester A

N

3

16

27

43

49

41

37

36

28

12

63

2014-15

Y

16

13

15

24

21

15

11

8

6

0

4

Semester B

N

28

13

37

52

38

53

33

25

19

13

39

2015-16

Y

11

10

19

34

38

25

17

11

12

2

9

Semester A

N

7

18

22

47

36

37

42

21

16

12

35

2015-16

Y

30

18

22

30

36

19

21

5

3

1

2

Semester B

N

15

17

31

57

51

44

43

30

5

1

19

2016-17

Y

10

10

23

35

19

13

7

7

3

1

12

Semester A

N

9

25

26

38

42

41

23

21

13

8

37

2016-17

Y

4

12

12

25

35

16

6

5

0

1

5

Semester B

N

12

21

34

45

65

50

29

16

8

8

42

2017-18

Y

9

11

10

9

31

16

10

1

3

1

6

Semester A

N

50

26

21

33

57

48

30

11

9

6

29

2017-18

Y

5

16

15

21

33

14

11

2

3

1

6

Semester B

N

21

42

34

39

90

58

52

28

14

1

48

95

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 4: Grades distribution for seven cohorts of CB2101
GradesYear/Cohort

PALSI

A+

A

A-

B+

B

B-

C+

C

C-

D

F

2014-15

Y

0

2

4

2

2

7

9

2

0

1

0

Semester A

N

14

28

42

43

73

49

19

18

13

9

11

2015-16

Y

19

23

16

27

30

22

14

11

6

1

4

Semester A

N

35

59

48

51

68

78

37

27

24

2

25

2015-16

Y

1

8

8

12

11

10

8

2

0

0

2

Semester B

N

23

35

56

81

72

53

36

18

6

9

22

2016-17

Y

23

31

31

46

42

22

11

7

0

1

8

Semester A

N

43

74

86

123

97

97

75

31

8

4

39

2016-17

Y

6

5

5

5

1

3

2

0

1

0

3

Semester B

N

26

32

58

73

62

51

28

13

9

5

21

2017-18

Y

25

21

20

26

26

3

2

2

2

0

1

Semester A

N

43

66

70

105

126

56

45

18

23

8

25

2017-18

Y

7

7

2

7

2

2

1

0

1

2

1

Semester B

N

29

45

68

63

36

41

34

20

8

9

21

Descriptive Statistics Information about Studies with “A Range” Students
In Figure 1, it shows that in seven out of eight cohorts PALSI students had a higher percentage
in “A Range” students than non-PALSI students for CB2100. In most of the cohorts, the PALSI
group has at least 5% more than the non-PALSI group with such substantial differences.
Similarly, Figure 2 shows that CB2101 has five out of seven cohorts PALSI students with thepercentage of the “A Range” students higher than the non-PALSI students. All the cohorts of
the PALSI students having “A Range” results are over 20%, and three cohorts are significantly
as high as 50%.
According to these two figures, it draws our attention that PALSI students’ contribution to “A
Range” is far more than the percentage than a normal distribution should have. Although not
all the cohorts were consistent to draw the same conclusion, it is believed that students who
had joined the PALSI scheme were aimed high and capable of achieving excellent academic
performance.

96

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Figure 1: CB2100 – “A Range” percentage of PALSI students and non-PALSI students

Figure 2: CB2101 – “A Range” percentage of PALSI students and non-PALSI students
Findings with t-test on PALSI and Non-PALSI Students’ Performance
The academic results are collected from 2014-15 to 2017-18, and the data include a total ofeight cohorts in CB2100 and seven cohorts in CB2101. In general, the data of different grades
would follow the normal distribution, and only the top tier percentile part represents “A Range”

97

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

performance. Nevertheless, the academic performances of PALSI students of both modules
have formed skewed distribution with a higher mean value than the normal distribution fromprevious observations. Further analyses for in-depth inferential statistical t-test are conducted
to investigate whether there is a significant difference in the GPA mean values of these twogroups. Relevant data of both modules are shown in Table 5 and Table 6 as below.
The performances are analysed by comparing their GPA mean values with the SPSS statisticalpackage (IBM) to show the mean difference. The p-values of the t-test for PALSI and nonPALSI students found that all eight cohorts of CB2100 are <.05, which represent their meansof these two groups are different with statistically significant results. In another module
CB2101, three cohorts have p-values <.05, and it is noted that those cohorts with p-values >.05
are relatively low PALSI enrolment numbers.
Table 5: Inferential statistics for studying CB2100
PALSI

Non-PALSI

Year / Cohort

Mean

N

SD

Mean

N

SD

p-value

2014-15 Semester A

2.55

135

1.27

2.23

355

1.25

.013*

2014-15 Semester B

3.09

133

0.91

2.60

350

1.21

<.001*

2015-16 Semester A

2.85

188

0.95

2.47

293

1.17

<.001*

2015-16 Semester B

3.11

167

0.74

2.80

313

0.96

<.001*

2016-17 Semester A

2.94

140

1.10

2.55

283

1.22

.002*

2016-17 Semester B

3.02

121

0.84

2.61

330

1.19

.001*

2017-18 Semester A

2.96

107

0.98

2.67

280

1.13

.02*

2017-18 Semester B

3.01

127

0.91

2.67

427

1.16

.002*

Note: *p < .05 statistically significant

Table 6: Inferential statistics for studying CB2101
PALSI

Non-PALSI

Year / Cohort

Mean

N

SD

Mean

N

SD

p-value

2014-15 Semester A

2.76

29

0.69

2.92

319

0.91

.35

2015-16 Semester A

3.10

173

0.88

2.93

454

1.02

.05*

2015-16 Semester B

3.03

62

0.80

2.95

411

0.98

.51

2016-17 Semester A

3.22

222

0.87

2.97

677

0.99

.001*

2016-17 Semester B

3.17

31

1.25

2.99

378

0.99

.34

2017-18 Semester A

3.51

128

0.67

3.03

585

0.95

<.001*

2017-18 Semester B

3.31

32

1.09

3.00

374

1.05

.11

Note: *p < .05 statistically significant

Results and Discussion
In general, the p-values of both modules support that there is a difference in the academic
performance of the PALSI group and non-PALSI group. In other words, students who havejoined the PALSI scheme could attain better performance in the accounting modules, after98

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019analysing the academic results with a series of the cohort. As mentioned in the former part,
HKDSE graduates might have taken the “Accounting” subject when they were studying insenior secondary school. According to the information provided by the Academic Regulations
and Records Office (ARRO), only their previous accounting knowledge in HKDSE of the
PALSI students could be identified. Thus, Table 7 shows the percentage of PALSI students
who have enrolled in HKDSE “Accounting” subject; the results show that around one-third ofthem had accounting knowledge before the undergraduate studies. In the light of that, anotherset of paired t-tests were run to investigate any performance advantages for PALSI students
with previous accounting qualification background.
Table 7: PALSI enrolments details from 2014 to 2017Year

No. of students enrolled in
PALSINo. of PALSI students
enrolled in HKDSE
Accounting subject% of all PALSI students
enrolled in HKDSE
Accounting subject

2014

297

93

31.3%

2015

590

184

31.2%

2016

514

178

34.6%

2017

394

107

27.2%

The endorsement form for joining the PALSI scheme signed from the colleges and thedepartments allowed us to collect academic results anonymously to perform data analysis forthe research purpose. Therefore, we are able to access the study background of PALSI students.
However, for those who did not participate in the PALSI scheme, the study background of nonPALSI students could not be analysed.
PALSI students’ performance of each module is further categorized into two smaller groups
based on their prior accounting experience. In Table 8 and Table 9, the two groups are named“HKDSE accounting = YES” and “HKDSE accounting = NO”, with a t-test on the mean values
comparisons.
Table 8: Comparison with different accounting background of CB2100 PALSI students
HKDSE accounting=YES

HKDSE accounting=NO

Year

Mean

N

SD

Mean

N

SD

p-value

2014-15 Semester A

2.98

39

1.00

2.38

96

1.33

.012*

2014-15 Semester B

3.46

47

0.77

2.88

86

0.92

<.001*

2015-16 Semester A

3.06

50

0.87

2.77

138

0.96

.065

2015-16 Semester B

3.32

56

0.76

3.00

111

0.71

.009*

2016-17 Semester A

3.34

52

0.75

2.71

88

1.21

.001*

2016-17 Semester B

3.38

25

0.57

2.92

96

0.88

.02*

2017-18 Semester A

3.16

30

0.54

2.88

77

1.10

.18

2017-18 Semester B

3.40

23

0.48

2.93

104

0.97

.02*

Note: *p < .05 statistically significant

99

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 9: Comparison with different accounting background of CB2101 PALSI students
Advanced accounting=YES

Advanced accounting=NO

Year

Mean

N

SD

Mean

N

SD

p-value

2014-15 Semester A

2.43

7

0.84

2.86

22

0.63

0.15

2015-16 Semester A

3.29

62

0.75

3.00

111

0.93

.04*

2015-16 Semester B

3.12

16

0.54

3.00

46

0.88

.62

2016-17 Semester A

3.47

86

0.58

3.06

136

0.99

.001*

2016-17 Semester B

3.06

15

1.45

3.27

16

1.08

.65

2017-18 Semester A

3.43

42

0.58

3.55

86

0.72

.34

2017-18 Semester B

3.60

12

0.95

3.15

20

1.16

.27

Note: *p < .05 statistically significant

From the analysis of the PALSI students’ performance, both accounting modules with a large
number of PALSI enrolments show that at least one sub-grade improvement is achieved. The
continuous study on different cohorts in four academic years – eight semesters proves that the
PALSI scheme contributes substantial enhancements in the academic performance ofaccounting disciplines. Those findings also provide supportive evidence that the PALSI
scheme is not solely for “high-risk” modules of the engineering undergraduate programmes(Im et al., 2017). Students with accounting training before are also interested in joining the
PALSI scheme in order to achieve higher academic performance.
Conclusions
In the City University of Hong Kong, the Peer Assisted Learning (PAL) scheme has been
implemented to support learning for those most challenging “high risk” subjects ofundergraduate candidates. Especially in various subjects of engineering and science
disciplines, the peer group learning approach has ensured that the learner-centered approach
could enhance the learning experience.
In this study, the accounting modules of the College of Business were selected to evaluate
academic performance. These core modules of thirteen undergraduate programmes provide
nearly two thousand quotas for every academic year. The PALSI scheme was introduced tojunior year students to provide extensive teaching and learning support. Such peer-to-peer
support opportunities have engaged students more interactively through the small class-sizeteaching mode.
Topping and Ehly (1998) state that collaborative learning experience is constructed through aseries of the understanding process – questioning, answering, discussing, clarifying, giving
examples, and receiving feedback. More interactive discussions during peer-assisted learning
sessions could offer additional opportunities to develop metacognition, which are different
from traditional tutoring sessions in higher education (Vygotsky, 1978). Peer tutoring with
supplemental instruction not only encourages students to be engaged in group discussions but
also turns them from passive listeners to active learners in collaborative learning. Researchers
have studied the different perspectives about education learning, Marton and Säljö (1976a,
1976b) point out that there are two kinds of learning development processes, which are“surface-level processing” and “deep-level processing”.

100

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

The findings from this longitudinal study in accounting modules show that increasing the time
given to accounting topics with the learner-centred approach can eventually help to enhance
the study performance. Although scaffolding the knowledge through peer learning could
achieve a good learning experience, it might require regular meeting schedules with anadequate attendance rate. Therefore, further studies on the relationship in attendance records
and academic performance would be our future analysis direction.

101

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
Bennett, S. R., Morris, S. R., & Mirza, S. (2018). Medical students teaching medical students
surgical skills: The benefits of peer-assisted learning. Journal of Surgical Education,
75(6), 1471–1474. https://doi.org/https://doi.org/10.1016/j.jsurg.2018.03.011
Burgess, A., McGregor, D., & Mellis, C. (2014). Medical students as peer tutors: a systematic
review. BMC Medical Education, 14(1), 115. https://doi.org/10.1186/1472-6920-14115
City University of Hong Kong. (2018, December 19). Bachelor’s Degree Programmes |Undergraduate Admissions. https://www.admo.cityu.edu.hk/jupas/entreq/bd
Duah, F., Croft, T., & Inglis, M. (2014). Can peer assisted learning be effective inundergraduate mathematics? International Journal of Mathematical Education in
Science and Technology, 45(4), 552–565.
https://doi.org/10.1080/0020739X.2013.855329
Herrmann-Werner, A., Gramer, R., Erschens, R., Nikendei, C., Wosnik, A., Griewatz, J., …
Junne, F. (2017). Peer-assisted learning (PAL) im medizinischen Grundstudium: eineÜbersicht. Zeitschrift Fur Evidenz, Fortbildung Und Qualitat Im Gesundheitswesen,
121, 74–81. https://doi.org/10.1016/j.zefq.2017.01.001
Hong Kong Examinations and Assessment Authority. (2018a, December 20). Development of
Public Exams in Hong Kong..
http://www.hkeaa.edu.hk/en/recognition/develop_hk_pub_exam/
Hong Kong Examinations and Assessment Authority. (2018b, December 20). Subject
Information.. http://www.hkeaa.edu.hk/en/hkdse/assessment/subject_information/
Im, S. W. T., Chiu, P. H. P., Shek, C. H., & Liu, D. Y. W. (2017). Performance evaluation ofcollege of science and engineering students with the PALSI scheme. International
Journal of Science, Mathematics and Technology Learning, 24(1), 27–34.
https://doi.org/10.18848/2327-7971/CGP/v24i01/27-34
Marrone, M., & Draganov, L. (2016). Peer assisted learning: Strategies to increase student
attendance and student success in accounting. In L. N. Wood & Y. A. Breyer (Eds.),
Success in higher education: Transitions to, within and from university (pp. 149–
165). https://doi.org/10.1007/978-981-10-2791-8
Marton, F., & Säljö, R. (1976a). On qualitative differences in learning: I - Outcome andprocess. British Journal of Educational Psychology, 46(1), 4–11.
https://doi.org/10.1111/j.2044-8279.1976.tb02980.x
Marton, F., & Säljö, R. (1976b). On qualitative differences in learning — II Outcome as afunction of the learner’s conception of the task. British Journal of Educational
Psychology, 46(2), 115–127. https://doi.org/10.1111/j.2044-8279.1976.tb02304.x
McLelland, G., McKenna, L., & French, J. (2013). Crossing professional barriers with peerassisted learning: Undergraduate midwifery students teaching undergraduate
paramedic students. Nurse Education Today, 33(7), 724–728.
https://doi.org/10.1016/j.nedt.2012.10.016
Meertens, R. (2016). Utilisation of a peer assisted learning scheme in an undergraduate
diagnostic radiography module. Radiography, 22(1), e69–e74.
https://doi.org/10.1016/j.radi.2015.08.004

102

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Qonda, M. (2017). Peer-assisted learning programme: Supporting students in high-risk
subjects at the mechanical engineering department at Walter Sisulu University.
Journal of Student Affairs in Africa, 5(2), 17–31.
https://doi.org/10.24085/jsaa.v5i2.2700
Rayner, G., & Papakonstantinou, T. (2018). The use of peer-assisted learning to enhancefoundation biology students’ understanding of evolution. International Journal of
Innovation in Science and Mathematics Education, 26(3), 65–77.
Riaz, I. (2014). Peer assisted versus expert assisted learning: A comparison of effectivenessin terms of academic scores. Journal of the College of Physicians and Surgeons-Pakistan : JCPSP, 24(11), 825–829. https://doi.org/11.2014/JCPSP.825829
Sudhakar, A., Tyler, J., & Wakefield, J. (2016). Enhancing student experience andperformance through peer-assisted learning. Issues in Accounting Education, 31(3),321–336. https://doi.org/10.2308/iace-51249
Supple, B., Best, G., & Pearce, A. (2016). “My purpose was to help them with accounting,not English”: An exploratory study of languages other than English in peer assisted
study sessions. Journal of Peer Learning, 9, 49–65.
https://ro.uow.edu.au/ajpl/vol9/iss1/6
Topping, K. J., & Ehly, S. W. (1998). Introduction to peer-assisted learning. In Peer-assisted
learning (pp. 1–23). Mahwah, N.J.: Routledge.
Vygotsky, L. (1978). Mind in society : the development of higher psychological processes.
M. Cole (Ed.). Cambridge, Mass: Harvard University Press.
West, H., Jenkins, R., & Hill, J. (2017). Becoming an effective peer assisted learning (PAL)
leader. Journal of Geography in Higher Education, 41(3), 459–465.
https://doi.org/10.1080/03098265.2017.1315384
Williams, B., Olaussen, A., & Peterson, E. L. (2015). Peer-assisted teaching: Aninterventional study. Nurse Education in Practice, 15(4), 293–298.
https://doi.org/10.1016/j.nepr.2015.03.008
Williams, B., & Reddy, P. (2016). Does peer-assisted learning improve academic
performance? A scoping review. Nurse Education Today, 42, 23–29.
https://doi.org/10.1016/j.nedt.2016.03.024
Corresponding author: Siu Wo Tarloff Im
Contact email: sw.im@cityu.edu.hk

103

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019104

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Evaluation of Self and Peer Assessments in a Second Year Engineering Module
David Hassell
University of Bath, United Kingdom
Kok Yueh Lee
Universiti Teknologi Brunei, Brunei Darussalam

105

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
This paper evaluates the use of peer and self-assessments as part of the learning process of anopen ended, essay-based course in a second-year degree engineering module in Brunei
Darussalam. The essays were marked using a rubric by the student, a peer, and the lecturer,
with students being pre-trained on the use of the rubric prior to the exercise. Comparison of themarks awarded by the different markers (student, peer, lecturer) showed that whilst there might
be correlations between different markers (i.e. peer - self; or lecturer - self) for marks on certain
sub-sections of the work, there was no overall correlation between marks for this open ended
problem. This lack of consistency highlights the subjective nature of marking essay-based
work, even with the use of a rubric. Feedback on the students’ experiences was obtained using
a questionnaire, and most students felt that the peer assessment exercise was a worthwhile
activity which aided both their learning and students’ motivation to learn. Analysis of student
performance in the exam, after the exercise, identified that almost all students did better in thequestion linked to the exercise than in others, further reinforcing this student view. The poor
mark concordance in this study indicates that both techniques are not suitable to quantitatively
evaluate student performance, however they had a positive impact on student learning. It is
recommended that this approach is incorporated in other open-ended assessments as a form offormative feedback with the provision of adequate tutor and student preparation.
Keywords: peer-assessment; self-assessment; rubrics; engineering; student experience; higher
education

106

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Higher Educational institutions increasingly view assessment within curriculum as methods
for learning rather than methods of measuring learning. Within the former, student, peer andself-assessment and the use of rubrics are favourably supported and encouraged in higher
education (Falchikov, 2003, 2005; Hanrahan & Isaacs, 2001) and are a useful technique for theevaluation of student performance and focal point for self-reflection. Asikainen, Virtanen,
Postareff, and Heino (2014) have suggested that long-term pedagogical training is not the only
way to develop university teaching and learning, and that the application of rubrics and peer
assessment can be an effective teacher-focused approach to improve student learning.
However, Liu and Carless (2006) have indicated that there is resistance to this shift fromlecturer assessment to peer assessment by both staff and students for reasons such as thereliability and fairness of peer assessment and the increasing the workload of lecturers as thepeer marks would have to be collated. Contextual factors such as students and lecturers in Hong
Kong perceiving assessment as the lecturer’s duty and students believing that lecturers are
more knowledgeable in assessing. To mitigate student reluctance, Sendziuk (2010) proposed
lecturers deliver written feedback and not award grades in assignments to ensure students
would value the feedback. Other approaches include involving students in creating theassessment criteria, the emphasis being the process of learning instead of performance.
Previous literature within engineering has focused on assessment of numerically basedquestions or problems (Davey, 2011), or student presentations (Montalvão & Baker, 2015),
and there is little research on the application of self and peer assessments using rubrics incomplex problem-based essays. This paper presents an evaluation of the use of peer and selfassessments using rubrics as part of the learning process for essay based open-ended
coursework in a module studied in year two of an engineering degree. The work investigates
the effectiveness of the peer-to-peer marking in accurately capturing the final student markwhen compared to lecturer marks, and investigates the impact of this assessment process on
student experience and learning.
The aims of this study were to:
1. Assess the capacity of engineering students to grade their own work and other students
work when provided with suitable training and a rubric marking scheme.
2. Determine the statistical correlation and concordance between the students’ peer andself-assessed marks and that of the lecturer.
3. Identify whether students valued this type of active self and peer assessment exercise
and in what ways they found it effective/ineffective using an anonymous and voluntaryestablished survey.
4. Identify any differences in performance and perception of the exercise based on courseof study.
Literature Review
Self-assessment refers to the assessment exercise where students evaluate their own work in
relation to their performance and ability or achievements (Andrade, 2010; Andrade & Boulay,
2003; Andrade & Valtcheva, 2009). It has been reported by Andrade (2010) that it enhanceslearning and achievement and aims to provide informative feedback on students’ achievements
in order to improve their abilities. In a similar manner, Nicol, Thomson, and Breslin (2014)
describe peer-assessment as a classroom exercise whereby students take on the role as theassessor to evaluate their peer’s work, either in the form of scoring, grading or throughfeedback. Arendt, Trego, and Allred (2016) stated that involving students in their own

107

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019assessment has been shown to improve students’ performance if implanted effectively while
Boud (1995) reported that both assessments engage students to be active participants in theirown learning.
Whilst some researchers such as Montalvão and Baker (2015) applied peer-assessment without
rubrics in what they described as a “holistic approach”, in general it is more common to provide
students with a marking rubric when applying peer and self-assessment. There is a large
amount of literature available outlining the use of rubrics over a wide range of disciplines andacademic levels (see for instance, Andrade, 2000; Andrade, Du, & Wang, 2008; Cho, Schunn,& Wilson, 2006; Moni & Moni, 2008; Tierney & Simon, 2004). Whilst there are still instancessuch as those reported by Reyna and Meier (2018) where inadequate rubrics are used, thebenefits of using rubrics in educational contexts have been reported by Andrade (2000, 2005),
Andrade and Du (2005), and Reddy and Andrade (2010) to be of help to students tocomprehend tutor’s expectations, understand the specific intended learning outcomes of theassignment or task, as well as providing feedback to students indicating their strengths andweaknesses.
Using rubrics during summative assessment poses multiple challenges for the lecturer,
including rubric reliability and validity (Andrade, 2005) and rater reliability (Moskal &Leydens, 2000). Work by Cho et al. (2006) looked at the reliability of peer review and foundthat it requires multiple peer assessors to be reliable. When evaluating the reliability of students
to accurately capture their performance in comparison to that of their lecturer, there areconflicting results reported within the literature. A recent review of literature (Brookhart, 2018)
reported that most literature claims a high inter-rater reliability, however the criteria for theseclaims are variable and whilst some studies report high correlation between lecturer and student
marking (see for instance, Freeman, 1995; Fry, 1990; Longhurst & Norton, 1997; Oldfield &Macalpine, 1995; Orpen, 1982) there are others that acknowledge low correlation (see for
instance, Kwan & Leung, 1996; Rushton, 1993). Low correlation suggests that lecturers andstudents have different interpretations towards the criteria set out in the rubrics indicating thatrubrics can be rather subjective. Both Andrade (2000), and Jonsson and Svingby (2007) havestated that a careful and well-designed rubric can promote students’ learning, enhancing theteaching and learning process whilst stimulating thinking processes. However other studies
have found no direct effect of rubrics on student performance (Tobajas, Molina, Quintanilla,
Alonso-Morales, & Casas, 2019) and to adequately capture the development of student
learning, rubric validity is important, with minimal student mark sensitivity based on theassessor. The lecturer also has to have a well-defined and transparent understanding anddefinition of what is required, with Rezaei and Lovorn (2010) reporting that they also need to
be well-trained when it comes to designing and implementing rubrics.
Within the field of engineering, Davey (2011) and Davey and Palmer (2012) found averagemarks awarded by assessor and assessee using rubrics were similar, although a closer look at
the results indicate considerable scatter. Later work by Davey (2015) reported that students
undertaking self-assessment of a mid-term test marked their work on average 16% higher than
the tutor did. The main differences were observed in only two of the five questions, andindicates that rater reliability is a consequence of the type of question posed as well as any
training raters receive. Recently work has been presented arguing that peer assessment can
provide similar marking to the lecturer (Rodgers, 2019), however the considerable scatter inthe results again indicate a danger when interpreting student marks through statistics. Theseseemingly conflicting findings illustrate the challenges faced when implementing rubrics andthe use of peer and self-assessments into the evaluation of student learning.

108

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Apart from assisting in the evaluation of students’ performances, rubrics can also be used for
self and peer assessments as part of formative assessment. Nagori and Cooper (2014) haveoutlined the usefulness of these techniques for formative assessment, where students are partial
assessors of their own work. In this, the emphasis is on the learning attainment rather thanscores or grades where the peer and self-assessments allow students to be judges of their own
work as well as of their peers. Whilst studies such as that presented by De Grez, Valcke, andRoozen (2012) and Davey (2015) have reported students expressing a positive opinion on theexperience, there are limited studies providing evidence of the approach leading to improved
performance in subsequent assessments.
Methodology
ParticipantsParticipants were from a Petroleum Engineering Bachelor’s degree (38 students) andFoundation degree in Process Engineering (58 students) who were all studying the same
module. The participants were selected based on convenience sampling as the students would
have undergone multiple assessment methods and were familiar with peer and self-assessment
exercises. The entry requirements of the students entering these two courses is different:
Foundation degree D, D in A-level mathematics and physics/chemistry; Petroleum Engineering
B, B, C in A-level mathematics, physics and chemistry. In both cohorts, almost all students
were nationals of Brunei Darussalam, with just one or two foreign students from Malaysia. All
96 participants volunteered to take part in the study and were allowed to withdraw from thestudy at any time without any negative implications. Ethical considerations to protect students’rights included completing an anonymised questionnaire and assessing their peers
anonymously.
Research Context
This study was carried out on a piece of coursework in a second-year engineering module,
Petroleum Refining, which consisted of 56 hours of content time between the lecturer andstudents over 14 weeks.
The coursework to be assessed was handed out to the students at the beginning of the semester
and took the form of a 600 word essay worth 10% of the final grade. The task involved
describing the various factors to be considered when locating an oil refinery. The coursework
handout is provided in Appendix A. The students were provided with six weeks in which tocomplete their work and once the students had finished the course, they were provided with aset of rubrics to evaluate their work. At the end of the module students undertook an exam
which was worth 70% of the total marks. As part of the exam, one section worth 15% evaluatedstudents’ understanding of the topics covered in the assignment, and this was used in part toevaluate the students’ retention of information relevant to the assignment.
Procedure
Both Jonsson and Svingby (2007), and Reddy and Andrade (2010) have indicated that student
use of a rubric must include an element of training for the student to understand its
implementation. In this study, students were given a half hour oral presentation anddemonstration prior to implementation of the rubric for self and peer assessments. The students
were then given one hour to mark two pieces of work using the rubric, their own (selfassessment) and that of one other student (peer assessment). For expediency, peer assessment
was facilitated by students collecting work from the front of the lecture hall, with each pieceof work being assessed by oneself and one peer marker. The reviewing process was

109

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

anonymised (no assessor names) and as a result students were unaware of who assessed theirwork. Whilst grading of work was mandatory, the provision of written feedback was madeoptional, and marking by the lecturer took place subsequent to this activity. The total process
took approximately two hours to complete, and was undertaken during a normal lecture period
for the module. The self and peer assessment marks, whilst provided to the students, did notcontribute to the overall mark of the module.
Of the 96 students from both cohorts, 59 participated in the self and peer assessment tasks (aresponse rate of 62%) and 65 students took part in completing the questionnaire (a response
rate of 68%). Six students did not complete the assessment forms. This highlights one limitationof the voluntary nature of the work, with students choosing whether to attend the session andsubmit their assessment for evaluation.
RubricA set of descriptive rubrics was distributed to students. The rubric included specific points thatshould have been made in the coursework, and is provided in Appendix B, along with the listof points used during assessment. Whilst it is normally the case that rubrics are handed out
with the assessment, they were not provided in this case as elements of the rubric included
specific answers which would undermine the assessment purpose. These points were included
to facilitate more accurate marking with the intention of reducing subjectivity caused byvariable markers. After marking, students were asked to return the assessment rubric to thelecturer for subsequent evaluation, and the lecturer marked all coursework with these marks
contributing towards the final student mark for the module. These marks were then comparedwith the results of self and peer assessment. Upon analysis it was found that some data were
missing. Not all peer and self-assessment forms were submitted and there were six omissions(all from Petroleum Engineering). This highlights that more attention should have been takenby the academic staff when receiving the assessment forms, and as a result there were slightly
more questionnaire responses than there were complete self and peer assessment pairs, with asummary provided in Table 1.
QuestionnaireThis study implemented a mixed methods approach for data collection and analysis. Data wascollected using both quantitative and qualitative methods. Once the assessment process was
finished, students were asked to complete an anonymised questionnaire seeking information toinvestigate students’ experiences of and attitudes towards rubrics and peer assessment. The
questionnaire is provided in Appendix C and comprised 21 statements of fixed-response typerating their agreement using a seven-point Likert scale (Likert, 1932). Ratings ranged from
Strongly Agree to Strongly Disagree. The questionnaire focused on students’ perception ofpeer marking to identify the extent to which students felt that they were adequately prepared
to assess others work based on a complex open-ended problem. Complementary andcontradicting statements were used in the questionnaire in order to check the validity andreliability of student’s response. For instance,
Q11. I took a serious attitude towards marking peers’ work.
Q12. I felt that I was critical of others when marking it.
were examples of complementary statements, whereas
Q17. I think that the rubrics was written in a clear manner which allowed me toaccurately assess the work.
Q20. I found the rubrics for the coursework confusing.
were examples of contradictory statements

110

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

There was also an optional, open-ended question where students were prompted to offer
additional comments on peer assessment. The responses were generally short, descriptivecomments reflecting their thoughts. Upon completion, students were then asked to return thequestionnaire results to the lecturer for analysis. A total of 65 questionnaires from both cohorts
were obtained (as shown in Table 1). The response rate of the questionnaire for the entire cohort
was between 67 and 72% for the Foundation degree cohort and 66 and 72% for the Petroleum
Engineering cohort, with not every student answering all questions in the questionnaire and not
all students providing comments for the optional open-ended question. Student comments on
their thoughts were coded into common themes: experiences of and attitudes towards rubrics;
and experiences of and attitudes towards peer assessment.
Table 1: Number of questionnaire responses, and self and peer assessment
marking for coursework.
CourseFoundation Degree (N=58)_
Petroleum Engineering (N=38)

Assessment paring
Male
Female
Total
13
29
42
10
7
17

Questionnaire responses
Total
42
23

(Note: Six students from Petroleum Engineering did not complete the assessment forms)

Analyses
Marks and responses were statistically evaluated using Excel software, and the mean andstandard deviation were calculated to quantify group responses. In addition, Spearman's
correlation coefficient was used to measure the strength and direction of association between
ranked variables. A value of 1 illustrated a perfect correlation between both variables, meaningthat an increase in one was found to indicate an increase in the other variable. A value of -1
was taken as a perfect anti-correlation between the variables, indicating that as one variable
increased, the corresponding response for the second variable decreased. As the significanceof the coefficient varies with sample size, results were taken to be statistically significant basedon the data provided in Zar (1984) using a significance level (α) of less than 0.025 unlessotherwise stated in the text.
Results and Discussion
Montalvão and Baker (2015) reported that students marked over a narrower distribution thanthe lecturer when undertaking peer assessment, with one conclusion drawn being that students
were reluctant to fail their peers. Comparison of the marks obtained in this study do not support
this view, with a minority of marks being in the failed range for both cohorts. Montalvão andBaker (2015) applied a “holistic approach” with a scale from 1 – 10 rather than a detailed rubric
as used in this study, and it is likely that this provision of a detailed framework on which toevaluate both themselves and peers led to increased confidence and understanding to provide
a fail mark.
Comparison of Marks Between Lecturer, Self and Peer Assessment
A comparison is presented in Table 2 between the marks awarded by each assessor using thelecturer mark as a reference. The data is presented as the difference between the average cohortmarks, calculated as the total marks allocated within the cohort divided by the number ofstudents, and then as the average variation in individual marks, calculated as the summed
difference between markers for each student divided by the number of students.

111

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 2: Difference in marks given to students as compared to the lecturer

GenderMale
FemaleAverage overall marks
for the cohort (%)

Average variation in individual marks
for the cohort (%)

Selfassessment

Peer
assessment

Self-assessment

Foundation Degree+10Petroleum Engineering

CoursePeer assessment

M

SD

M

SD

+20

21.2

21.8

26.7

20.4

+3

+4

24.2

24.6

30.0

22.9

Foundation Degree

+7

+7

24.1

20.2

28.2

25.6

Petroleum Engineering

+30

+30

39.8

20.8

31.1

22.4

When looking at the average overall marks of the cohort, it can be seen that the total numberof marks awarded for Petroleum Engineering students is similar for all assessment types whilst
marks given for self and peer assessment are much greater for the Foundation degree students.
At a rudimentary level this might indicate that the self and peer assessment gave an accurate
indication of the cohort average performance for Petroleum Engineering students. However, amore detailed comparison for individual students indicates that the average difference between
assessors was over 20% with Foundation degree students actually being slightly closer to thelecturer mark than the Petroleum Engineering students. The cohort averages for selfassessment indicate that students were less generous with themselves (excepting female
Petroleum Engineers) when compared to peer-assessment, with the average difference being
three to ten percent higher than the lecturer. Interestingly, peer assessment marks weregenerally higher than self-assessment apart from female Petroleum Engineers where there wereno major differences between their self and peer marks. The average variation in individuallyawarded marks is considerably larger than the difference in the cohort average, and this is best
represented for the case of Petroleum Engineering male students. The average overall marks
agree within 4% for both self and peer assessments, whilst the average absolute difference
between the marking is closer to 24 – 30%. It is important to represent the results as criticallyas possible, as any future use of the techniques evaluated can affect individual student
attainment and its measurement. This was also observed in by Davey (2015), although not
explicitly stated, who reported agreement in terms of question averages but comparison of thedata illustrated significant average difference between markings similar to this study.
Spearman Ranks were calculated for the individual cohorts, and no statistically relevantcorrelation was observed for the Petroleum Engineering students, perhaps as a result of theirsmall cohort sizes. Figure 1 provides the ratio of self/peer to lecturer marks (scaled) to against
the normalised lecturer mark with respect to the highest mark awarded for self-assessment,
peer assessment and ratio of self to peer mark. Amongst Foundation Engineering students,
there are negative correlations for both peer and self for both male and female students[Spearman Ranks: male self (-0.898), male peer (-0.623), female self (-0.604), and female peer(-0.615)]. This indicates that using the lecturer mark as a benchmark of student performance,
students who performed weakly in the assessment were more likely to be over-marked by boththemselves and their peers.

112

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Foundation Degree in Process
Engineering

Petroleum Engineering degree

Note: The triangles (D) represent female students and circles (¡) represent males

Figure 1: Comparisons of lecturer marks awarded to students from self, peer and lecturer
assessment. Unless explicitly stated in the figure, scaled marks are those divided by thelecturer mark for that student.
No correlation was observed in Figure 1 for the ratio of self to peer mark plotted against lecturermark, and Figure 2 presents a comparison of the same ratio plotted against normalised peer
mark.

113

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Foundation Degree in Process
Engineering

Petroleum Engineering degreeMarks are plotted against peer mark normalised with respect to the highest peer markgiven
Note: The triangles (D) represent female students and circles (¡) represent males

Figure 2: Comparison of self to peer marks. Scaled marks are those divided by the peer
assessment mark for that student.
A negative correlation for the Foundation students [Spearman Ranks: male (-0.633), female (0.435, significant for α = 0.05)] is observed, indicating that as the peer mark decreases, students
have a tendency to mark themselves more highly in self-assessment. This could indicate thatweaker students tend to overestimate their own ability in comparison to others. The results in
Figure 1 also indicate that higher scoring students were given lower marks by peers in thisstudy, when compared to marks given to the lower scoring students. However, students are
more likely to base their perception on student ability on the historical performance throughout
the degree rather than unknown performance in an individual assessment.
Self and lecturer marking

Lecturer and peer marking

Marks are plotted against the normalised student degree mark for the student submitting the assessment These
results are for those students undertaking the Foundation Degree in Process Engineering

Figure 3: Comparison of marks between self and lecturer and lecturer to peer. Scaled marks
are those divided by the lecturer mark for that student.

114

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Figure 3 presents results for Foundation students of self and peer assessment, marks scaled bylecturer mark against their normalised performance in their overall degree. It indicates the level
to which the students over or under mark themselves with respect to actual performance as
determined by the lecturer. The correlation factors for these results are relatively low[Spearman Rank: peer (0.340) and self (0.255)] however, they could still be considered
statistically significant for the sample size (n=40) at higher α values of 0.05 and 0.1 respectively
and indicate a positive correlation between both self and peer assessment marks and student
degree performance.
Whilst there is considerable scatter in the data, this positive correlation indicates that asacademic performance (as measured by performance in overall degree) increases, both
perception of performance by self and peer increases with respect to actual performance forstudents. In a study in the United States of America, Sadler and Good (2006) reported thatpoorly performing students have the tendency to overrate themselves as compared to tutor’s
grading, and this is different from the findings presented here. One possibility for the difference
is as a result of the cultural background of the students involved in this work. Unlike in someWestern societies, anecdotal evidence and experience indicates that students in South East Asia
are more likely to view good academic performance with admiration rather than jealousy. As
a result, there will not be the same negative perception of good performance, akin to that seen
with “rate-busting” in western industries, and hence the students in this study are biased in the
opposite manner, that is, towards rather than against high performing students.
It is noted that the quantitative results presented here have similarities to previous studies. The
results concur with Davey (2015) with very poor agreement in marks between assessors andthere is significant scatter in data as seen by Davey and Palmer (2012). The results also indicate
poor rater reliability similar to Andrade (2005) and Moskal and Leydens (2000). As with
Andrade and Du (2005), there was no obvious gender bias in the poor correlation, and students
provide on average higher marks for both self-assessment (as with Davey, 2015) and De Grezet al., 2012) and peer-assessment (as with Montalvão and Baker, 2015). However, the analysis
presented here are more scattered than for Davey (2015) indicating that self-assessment is more
unreliable when compared to lecturer assessment when the assessment is based on open ended
rather than closed questions.
Even with a rubric to guide students as in this work, or model answers as in the case of Davey(2015), students seem to be on average more generous than the lecturer. As a result, it seems
that to maintain fair and consistent marking, all students in a cohort should be marked by thesame individual or group of individuals. These large differences contradict the assertion by
Asikainen et al. (2014) that when using peer assessment, only assessment marking with
substantially different marks given by the assessors should require reassessment by the teacher
to ensure reliability.
To see whether discrepancies in marking were caused by a specific element of the rubrics, themarks between each assessor (lecturer, self and peer) for the total marks were compared within
the four categories: Presentation, Introduction, Discussion, Conclusion. Figure 4 presents a
detailed evaluation of the similarity in marking between various assessors with the observed
difference scaled with respect to maximum positive or negative deviation possible.

115

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Foundation Degree in Process
Engineering

Petroleum Engineering degree

Difference is scaled with respect to maximum positive or negative deviation possible. Zero indicates that bothassessors provided the same score.

Figure 4: Comparison of marks between assessors for the different sections
The degree to which marks between assessors were similar for the different sections of the
rubric for Foundation degree and Petroleum Engineering is represented on the x-axis, where
for a value of zero both assessors provided the same score. The result indicates that whilst there
was a relatively high agreement on marks of individual sections between assessors, equivalent
to 70% – 90% exact agreement in some cases, these were not carried forth into consistently
high agreement in the total marks awarded. The sections with the highest level of agreement
between assessors tended to be the presentation or introduction sections, but quantitatively themarks were not consistent between lecturer and either self or peer assessments. This poorcorrelation between different markers highlights the challenge in the development of athorough methodology for the implementation of robust and repeatable student self and peer
assessments. A past study in a Chemical Engineering class by Davey (2011) found that students
generally gave higher marks for descriptive questions as compared to the lecturer, whilst bothstudents and lecturer marked the numeric questions equally on average. The development of
rubrics, even in non-technical disciplines, requires a great deal of effort and time and in thiswork, the very poor agreement in areas that require a higher level of technical knowledge are116

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019likely attributed to a lack of in-depth knowledge by students on the arguments surrounding
each technical point.
Hassmén, Sams, and Hunt (1996) found that students who undertook self-assessments
performed better in the final test, whilst Andrade and Du (2005) stated that students’
perceptions were that they attain better results or grades as a result of the use of rubrics. To
compare the impact in this study, a comparison of the total marks awarded in the final exam isgiven in Figure 5 against the marks awarded for the question related to the topic covered in thecoursework. As can be seen, both cohorts on average performed significantly better in thecoursework related question than in the overall exam. Whilst this indicates that the coursework
had long term impact on the students learning and recall of information, it should be noted thatthe improved performance could also be due to the level of the questions in the exam or beindicative that students found those topics easier to learn than others. It cannot be definitively
proven which of the above arguments resulted in the increased student performance, however
the results when combined with student feedback in Table 3 do suggest that at the very least,
the use of rubric has not hindered the students learning as shown by subsequent assessments.
The overall results of this study agree with those of others such as Falchikov and Boud (1989)
and indicate that even though there is a lack of agreement between students and tutor’s marks,
self-assessment and peer assessment are valuable assessment tools by providing feedback tostudents on their learning and educational standards. Through undertaking the marking
themselves they are learning through active practice, rather than passive participation, andtherefore more likely to remember the information over a longer timeframe.
Foundation Degree in Process
Engineering

Petroleum Engineering degree

The dotted line in both graphs represents the x = y line

Figure 5: Comparisons of the exam mark against related question markStudents’ Experience of Learning
In addition to the correlation of rubrics with lecturer, self and peer marks, this study was also
concerned with the students’ perceptions of and the attitudes towards the peer and selfassessment exercise. Table 3 provides information on the student responses to the questionnaire
survey along with cohort averages and standard deviations for each question. In subsequent
sections these averages will be presented in square brackets to support interpretation of
students’ experiences.

117

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 3: Summary of number of questionnaire responses and category of response for eachsurvey statement. Numbers in brackets are the number of students who filed a response.
Foundation
Degree (n)Survey Statement Response
Q.

1

2

3

4

5

6

7

1)

7

5

3

4

11

26

6

M

Petroleum
Engineering (n)
SD

M

TOTAL
(n)
SD

M

SD

4.21
2.13
5.70
0.82
4.76 (62) 1.90
(39)
(23)
2)
0
0
2
11 26 17 6
5.25
0.95
5.18
1.01
5.23 (62) 0.97
(40)
(22)
3)
0
1
5
12 15 20 12 5.43
1.19
5.04
1.40
5.29 (65) 1.27
(42)
(23)
4)
1
2
4
21 12 20 5
5.12
1.17
4.39
1.44
4.86 (65) 1.31
(42)
(23)
5)
0
0
1
12 22 20 10 5.50
1.04
5.22
0.95
5.40 (65) 1.01
(42)
(23)
6)
0
0
3
24 18 12 6
4.85
1.05
5.00
1.13
4.90 (63) 1.07
(40)
(23)
7)
2
3
11 22 11 9
5
4.23
1.46
4.52
1.38
4.33 (63) 1.43
(40)
(23)
8)
11 12 17 22 2
0
1
3.05
1.10
2.74
1.51
2.94 (65) 1.26
(42)
(23)
9)
0
0
1
7
27 23 7
5.52
0.92
5.26
0.81
5.43 (65) 0.88
(42)
(23)
10) 0
0
3
20 20 18 4
5.14
1.05
4.74
0.92
5.00 (65) 1.02
(42)
(23)
11) 0
0
4
14 15 17 13 5.17
1.19
5.67
1.24
5.33 (63) 1.22
(42)
(21)
12) 0
0
4
19 22 15 5
4.81
1.02
5.26
1.05
4.97 (65) 1.05
(42)
(23)
13) 1
3
8
12 22 10 9
4.86
1.30
4.70
1.66
4.80 (65) 1.43
(42)
(23)
14) 0
0
1
7
28 19 10 5.43
0.89
5.52
1.04
5.46 (65) 0.94
(42)
(23)
15) 0
0
0
5
15 23 20 5.88
0.88
6.00
1.04
5.92 (63) 0.94
(40)
(23)
16) 0
0
0
4
14 23 19 5.90
0.88
6.05
0.97
5.95 (60) 0.91
(39)
(21)
17) 0
0
2
5
22 24 12 5.55
0.89
5.70
1.15
5.60 (65) 0.98
(42)
(23)
18) 0
0
1
7
20 26 11 5.52
0.77
5.74
1.21
5.60 (65) 0.95
(42)
(23)
19) 1
1
3
19 20 15 5
5.05
1.01
4.59
1.47
4.89 (64) 1.20
(42)
(22)
20) 5
10 8
25 9
2
1
3.53
1.37
3.59
1.33
3.55 (60) 1.35
(38)
(22)
21) 3
6
8
23 12 6
3
4.00
1.34
4.18
1.59
4.07 (61) 1.42
(39)
(22)
Note: Seven point Likert scale: 7 = strongly agree, 1 = strongly disagree, and 4 = no opinion or neutral response.

118

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Statements:
1. I have undertaken peer assessment previously.
2. Peer assessment is a worthwhile activity.
3. Giving feedback to my peers is very difficult.
4. I feel that my peers have adequate knowledge to evaluate my work.
5. I feel that peer assessment is helpful to my learning.
6. Giving feedback to my peers is useful to me.
7. I would prefer not to do peer assessment on others.
8. I prefer peer assessment rather than lecturer's feedback.
9. I learnt something through performing peer review.
10. Peer assessment activity motivates me to learn.
11. I took a serious attitude towards marking peers' work.
12. I felt that I was critical of others work when marking it.
13. When marking the coursework, I focused on the task at hand, and did not find mymind wandering to how my own coursework was being marked.
14. I learnt something further about the subject through marking coursework using the
rubrics.
15. Having used the rubrics, I now have a better understanding of what was expected in
the coursework
16. Having marked coursework using a rubrics, I now have a better understanding ofwhat is expected of me during coursework.
17. I think that the rubrics was written in a clear manner which allowed me to accurately
access the work.
18. I think that the presentation given at the beginning of the class on how to use rubrics
prepared me well enough to assess the work.
19. I think rubrics based peer assessment is a fair method to assess student's performance.
20. I found the rubrics for the coursework confusing.
21. I would like more assessment run in this manner in the future.
Experiences of and Attitudes Towards Rubrics
Previous research (Andrade & Du, 2005; Davey & Palmer, 2012; De Grez et al., 2012) has
shown that students’ perceptions on using rubrics are largely positive, and this was the generaltrend observed in this work. Students from both cohorts reported that they found the rubric well
written [Q17: FD 5.55, PE 5.70] and did not find it confusing [Q20: FD 3.53, PE 3.59]. They
also felt that the presentation and training at the beginning of the session prepared them wellenough to implement the rubrics [Q18: FD 5.52, PE 5.74]. This indicates that despite thedifferences between assessor marks results presented previously, students felt that they wereadequately prepared to assess their work using the provided rubric. Students also felt that the
rubrics is a fair assessment method to assess students’ performance [Q19: FD 5.05, PE 4.59].
This is highlighted by one student, as shown in the following extract.
It is nerve wrecking but at least I know why I lost and won some marks.
Davey (2015) found that the use of rubrics stimulated interest in the course. Students from bothcohorts felt that by marking their coursework using rubrics enhanced their understanding of
the module content [Q14: FD 5.43, PE 5.52], and students also found that the use of the rubrics
improved their understanding of what was required for the coursework [Q15|Q16: FD
5.88|5.90, PE 6.00|6.05]. This is further supported by students’ additional comments in thequestionnaire, as shown in the following extracts below.

119

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Rubric is very useful in the sense that we know what points we should raise whilewriting our report.
Interesting way of marking the coursework. Looking forward for the methods results.
Fun activity.
It was fun and a new experience for me.
An increased understanding of coursework expectations through the use of the rubric is an
expected response, as students were provided with a detailed written description of how the
work was to be marked alongside the demonstration of how to implement it. The use of the
rubrics encourages students to learn more about the content of the module as they are motivated
to at the very least on a superficial level to understand the content relating to the coursework
in order to assess their peers. Andrade (2010) stated that rubrics provide informative feedback
on students’ knowledge and competency, and this is further illustrated in the results presented
here.
Experiences of and Attitudes Towards Peer Assessment
The previous experience of students in this study to peer assessment was varied [Q1: FD 4.21,
PE 5.70] with a greater proportion of the Petroleum Engineering cohort having undertaken it
previously to the Foundation degree cohort. The results in Table 3 highlight that students inboth cohorts found the use of peer assessment to be a useful activity [Q2: FD 5.25, PE 5.18]
which aided their learning [Q5|Q9: FD 5.50|5.52, PE 5.22|5.26]. Students also found the peer
assessment exercise motivated them to learn [Q10: FD 5.14, PE 4.74]. A previous study by
Nicol et al. (2014) found that peer assessment is favoured positively by students and this isfurther illustrated in this work, and the students from both cohorts also felt that their peers had
adequate knowledge to assess their them [Q4: FD 5.12, PE 4.39] and that they took on aprofessional attitude in doing the exercise [Q11|Q12|Q13: FD 5.17|4.81|4.86, PE
5.67|5.26|4.70]. Despite this, some students voiced concerns or doubts on peer assessment as
noted by some students in the following extracts:
Some people do not have enough knowledge to actually give extra point for additional
points.
I need to refer to someone more expert when marking other’s papers in order tominimise the errors during marking.
I need more experience when marking others/papers as wrong way of marking might
disrupt others’ outcome.
I am concern on how people will see the way I mark their paper.
Bolton (2006) found that the use of rubrics can be useful in providing feedback to students,
whilst Andrade (2005) have stated that it is not a direct replacement for instruction, lecturer
feedback and opportunities to ask questions. These findings are consistent with the response of
students during this work, and whilst both student cohorts find providing feedback to others
useful [Q6: FD 4.85, PE 5.00], they did not view it as an acceptable replacement for lecturer
feedback [Q8: FD 3.05, PE 2.74]. Unfortunately, it cannot be evaluated through their responses

120

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

whether or not the students realised that through applying the rubric to their and others’ work
they are obtaining much more detailed indirect feedback from the lecturer on both their workand that of others. Davey (2011) reported broad student agreement that peer assessment is aneffective way to learn and stimulates interest in course material although idealised solutions of
the lecturer were essential for successful peer assessment outcomes. Preferences for lecturer’s
feedback was also apparent as expressed by some of the students in their comments, illustrated
by the following from three different students:
I would like the lecturer to mark it first, so that we could compare our marking with thelecturer straight away.
I prefer if the lecturer marks our report because they have deeper understanding on
this module/topic of the report.
I prefer the hands of experts to mark the coursework.
In spite of students highlighting that the activity increased their learning and understanding ofboth the coursework expectations and course content, they acknowledged that giving feedback
to peer was a difficult task [Q3: FD 5.43, PE 5.04] and they weakly indicated that they would
rather not do it [Q7: FD 4.23, PE 4.52] and were non-committal about the approach being
adopted in other modules [Q21: FD 4.00, PE 4.18]. This result is different from student
perceptions reported by Davey (2015) for self-assessment at an Australian University whoreported using a similar Likert scale that students were keen to have in other courses [average4.7]. This indicates the challenges present in applying teaching methodologies across different
cultures and provides data contrary to the views presented in Richmond (2007) that Asianstudents are willing to move away from the spoon-fed teaching culture which they have been
traditionally subjected to and embrace other forms of learning. One student also suggested to
be given more time in the peer assessment activity as 90 minutes can be rather tight when they
had to do a self-assessment followed by a peer assessment.
Given the results presented in both the marks and questionnaire analysis, the students seem tohave been unknowingly ill-prepared to accurately evaluate their work. One issue is that a ‘nonexpert’ might have been awarding marks when the listed relevant points were raised in the
essay, even when that argument point was presented incorrectly. Subsequent informal
discussion with students highlighted that when evaluating the points in Appendix B, there weredifferences in interpretation depending on the context surrounding where in the work the point
was raised. The technical expert caught numerous examples where the points raised met theinitial rubric criteria however they were presented in the wrong context (specifically Discussion
points 6-9). This level of subjectivity to the marking highlights a major failing with the mannerin which the rubric was presented, with additional detail being required for each point to allow
the students to accurately identify its relevant inclusion. Whilst a more detailed rubric might
reduce ambiguity, it is unlikely that the assessor can foresee every argument in an open-ended
problem. Indeed it was for this reason that the rubric allowed for “additional points”. From apractical perspective, a balance is required between the need to develop robust and thoroughrubrics for the assessment of work and the competing demands of lecturer time.
Conclusion
The work presented in this study presents an evaluation of the use of peer and self-assessments
as part of the learning process of an open-ended essay-based coursework in a year two degree

121

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019engineering module in Brunei Darussalam. The present study sets out to investigate whether,
given reasonable training on rubrics, students were capable to assess their own work as well asof their peers; the correlation between the assessed work (self, peer and lecturer) and students’
perceptions of the self and peer assessment exercise. The study was undertaken on two different
cohorts of students, and comparison of the marks awarded by different markers; student; peer;
and lecturer; showed very poor correlation between marks. Whilst there were correlations
between different markers (i.e. peer – self) for certain subsections of the work, these tendered
to be for the non-technical sections and there was no overall correlation between marks.
Correlation for the technical components was very poor, and likely the result of the in-depth
understanding required to accurately apply the rubric to the context surrounding the various
points required in the coursework. Comparison between student marking and lecturer founddifferences to those observed elsewhere, indicating that cultural background impacts how
students perceive themselves and others with respect to academic ability and their use of rubrics
to evaluate others work. Student feedback to the exercise found that most students held positive
attitudes towards peer-assessment and felt that peer-assessment was a worthwhile activity
which aided both their learning and motivation to learn. Bruneian students were open to being
critically evaluated by their peers and appreciated the opportunity to experience new
approaches and methodologies in the classroom. However, they then contradicted themselves
somewhat by indicating that they would have preferred not to have to do it, perhaps as a resultof finding it a difficult process to go through. This indicates the difficulty in introducing new
pedagogical practices into the classroom, where negative student feedback to difficult
situations is in conflict with the positive learning experience. Whilst the poor markconcordance in this exercise indicates that both peer and self-assessment are not suitable toquantitatively evaluate student performance, the overall student experience was seen as
positive and motivational to their learning and suitable for formative assessment.
The results of the present study suggest that more training is necessary for students in using
rubrics as part of their assessment. A crucial message lies in the validity and reliability of
rubrics having an impact on the quality of peer and self-assessments. This study only involved
one second year class student undertaking a specific module within a programme, thislimitation should be taken into account when considering the extent to which the results can begeneralised into the wider higher education contexts. Although careful consideration was
accounted for with respect to interrater reliability, the rater found that there is still a requirement
for comprehensive training to increase the accuracy of rating. The implementation of rubrics
in higher education has always been contextual, challenging and subjective. Peer and selfassessments can be alternatives to assessment along with the use of rubrics, provided all aspects
are planned methodically and students are made conscious of the steps and procedures in order
to execute it well.

122

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
Andrade, H. G. (2000). Using rubrics to promote thinking and learning. Educational
Leadership, 57(5), 13–19.
Andrade, H. G. (2005). Teaching with rubrics: The good, the bad, and the ugly. College
Teaching, 53(1), 27–31. https://doi.org/10.3200/CTCH.53.1.27-31
Andrade, H. G. (2010). Students as the definitive source of formative assessment. In H,
Andrade & G. Cizek, (Eds.). Handbook of Formative Assessment, (pp. 90–105).
Florence, KY: Routledge.
Andrade, H. G., & Boulay, B. A. (2003). Role of rubric-referenced self-assessment inlearning to write. Journal of Educational Research, 97(1), 21–30.
https://doi.org/10.1080/00220670309596625
Andrade, H. G., & Du, Y. (2005). Student perspectives on rubric-referenced assessment.
Practical Assessment, Research & Evaluation, 10(3), 1–11.
Andrade, H. G., Du, Y., & Wang, X. (2008). Putting rubrics to the test: The effect of a model,
criteria generation, and rubric‐referenced self‐assessment on elementary school
students’ writing. Educational Measurement: Issues and Practice, 27(2), 3–13.
https://doi.org/10.1111/j.1745-3992.2008.00118.x
Andrade, H. G., & Valtcheva, A. (2009). Promoting learning and achievement through selfassessment. Theory into Practice, 48(1), 12–19.
https://doi.org/10.1080/00405840802577544
Arendt, A., Trego, A., & Allred, J. (2016). Students reach beyond expectations with cafeteriastyle grading. Journal of Applied Research in Higher Education, 8(1), 2–17.
https://doi.org/10.1108/JARHE-03-2014-0048
Asikainen, H., Virtanen, V., Postareff, L., & Heino, P. (2014). The validity and students’experiences of peer assessment in a large introductory class of gene technology.
Studies in Educational Evaluation, 43, 197–205.
https://doi.org/10.1016/j.stueduc.2014.07.002
Bolton, F. C. (2006). Rubrics and adult learners: Andragogy and assessment. Assessment
Update, 18(3), 5–6. https://doi.org/10.1002/au.183
Boud, D. (1995). Assessment and learning: contradictory or complementary? Assessment,
35–48.
Brookhart, S. M. (2018). Appropriate criteria: Key to effective rubrics. Frontiers in
Education, 3(22). http://doi.org/10.3389/feduc.2018.00022
Cho, K., Schunn, C. D., & Wilson, R. W. (2006). Validity and reliability of scaffolded peer
assessment of writing from instructor and student perspectives. Journal of
Educational Psychology, 98(4), 891–901. http://dx.doi.org/10.1037/00220663.98.4.891
Davey, K. R. (2011). Student peer assessment: Research findings from a case study in amaster of chemical engineering coursework-program. Education for Chemical
Engineers, 6(4), e122–e131. https://doi.org/10.1016/j.ece.2011.08.004
Davey, K. R. (2015). Student self-assessment: Results from a research study in a level IV
elective course in an accredited bachelor of chemical engineering. Education for
Chemical Engineers, 10, 20–32. https://doi.org/10.1016/j.ece.2014.10.001

123

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Davey, K. R., & Palmer, E. (2012). Student peer assessment: A research study in a level III
core course of the bachelor chemical engineering program. Education for Chemical
Engineers, 7(3), e85–e104. https://doi.org/10.1016/j.ece.2012.04.001
De Grez, L., Valcke, M., & Roozen, I. (2012). How effective are self- and peer assessment oforal presentation skills compared with teachers’ assessments? Active Learning in
Higher Education, 13(2), 129–142. https://doi.org/10.1177%2F1469787412441284
Falchikov, N. (2003). Involving students in assessment. Psychology Learning & Teaching,3(2), 102–108. https://doi.org/10.2304%2Fplat.2003.3.2.102
Falchikov, N. (2005). Improving assessment through student involvement: Practical solutions
for aiding learning in higher and further education. Oxon, UK: Routledge.
Falchikov, N., & Boud, D. (1989). Student self-assessment in higher education: A metaanalysis. Review of Educational Research, 59(4), 395–430.
https://doi.org/10.3102%2F00346543059004395
Freeman, M. (1995). Peer assessment by groups of group work. Assessment & Evaluation inHigher Education, 20(3), 289–300. https://doi.org/10.1080/0260293950200305
Fry, S. A. (1990). Implementation and evaluation of peer marking in higher education.
Assessment and Evaluation in Higher Education, 15(3), 177–189.
https://doi.org/10.1080/0260293900150301
Hanrahan, S. J., & Isaacs, G. (2001). Assessing self-and peer-assessment: The students’views. Higher Education Research & Development, 20(1), 53–70.
https://doi.org/10.1080/07294360123776
Hassmén, P., Sams, M. R., & Hunt, D. P. (1996). Self-assessment responding and testing
methods: effects on performers and observers. Perceptual and Motor Skills,
83(3_suppl), 1091–1104. https://doi.org/10.2466/pms.1996.83.3f.1091
Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity andeducational consequences. Educational Research Review, 2(2), 130–144.
https://doi.org/10.1016/j.edurev.2007.05.002
Kwan, K., & Leung, R. W. (1996). Tutor versus peer group assessment of student
performance in a simulation training exercise. Assessment & Evaluation in Higher
Education, 21(3), 205–214. https://doi.org/10.1080/0260293960210301
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology.
Liu, N.-F., & Carless, D. (2006). Peer feedback: the learning element of peer assessment.
Teaching in Higher Education, 11(3), 279–290.
https://doi.org/10.1080/13562510600680582
Longhurst, N., & Norton, L. S. (1997). Self-assessment in coursework essays. Studies in
Educational Evaluation, 23(4), 319–330.
Moni, R. W., & Moni, K. B. (2008). Student perceptions and use of an assessment rubric fora group concept map in physiology. Advances in Physiology Education, 32(1), 47–54.
https://doi.org/10.1152/advan.00030.2007
Montalvão, D. & Baker, T. (2015). Correlating peer and tutor assessment on a low-stakes
engineering assignment. International Journal of Mechanical Engineering Education,
43 (3), 168-179. https://doi.org/10.1177/0306419015603008

124

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: Validity and reliability.
Practical Assessment, Research & Evaluation, 7(10), 71–81.
Nagori, R., & Cooper, M. (2014). Key principles of peer assessments: A feedback strategy toengage the postgraduate international learner. IAFOR Journal of Education, 2(2).
https://doi.org/10.22492/ije.2.2.07
Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher
education: a peer review perspective. Assessment & Evaluation in Higher Education,
39(1), 102–122. https://doi.org/10.1080/02602938.2013.795518
Oldfield, K. A., & Macalpine, J. M. K. (1995). Peer and self‐assessment at tertiary level - anexperiential report. Assessment in Higher Education, 20(1), 125–132.
https://doi.org/10.1080/0260293950200113
Orpen, C. (1982). Student versus lecturer assessment of learning: a research note. Higher
Education, 11(5), 567–572. https://doi.org/10.1007/BF00194421
Reddy, Y. M., & Andrade, H. G. (2010). A review of rubric use in higher education.
Assessment & Evaluation in Higher Education, 35(4), 435–448.
https://doi.org/10.1080/02602930902862859
Reyna, J., & Meier, P. (2018). Learner-generated digital media (LGDM) as an assessment
tool in tertiary science education: A Review of Literature. IAFOR Journal of
Education, 6(3). https://doi.org/10.22492/ije.6.3.06
Rezaei, A. R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment throughwriting. Assessing Writing, 15(1), 18–39. https://doi.org/10.1016/j.asw.2010.01.003
Richmond, J. E. D. (2007). Bringing critical thinking to the education of developing country
professionals. International Education Journal, 8(1), 1–29.
Rodgers, T. L. (2019). Peer-Marking and peer-feedback for coding exercises. Education for
Chemical Engineers, 29, 56–60. https://doi.org/10.1016/j.ece.2019.08.002
Rushton, C. (1993). Peer assessment in a collaborative hypermedia environment: A casestudy. Journal of Computer-Based Instruction, 20(3), 75–80.
Sadler, P. M., & Good, E. (2006). The impact of self-and peer-grading on student learning.
Educational Assessment, 11(1), 1–31. https://doi.org/10.1207/s15326977ea1101_1
Sendziuk, P. (2010). Sink or swim? Improving student learning through feedback and selfassessment. International Journal of Teaching and Learning in Higher Education,
22(3), 320–330.
Tierney, R., & Simon, M. (2004). What’s still wrong with rubrics: focusing on theconsistency of performance criteria across scale levels. Practical Assessment,
Research & Evaluation, 9(2), 1–10.
Tobajas, M., Molina, C. B., Quintanilla, A., Alonso-Morales, N., & Casas, J. A. (2019).
Development and application of scoring rubrics for evaluating students’ competencies
and learning outcomes in Chemical Engineering experimental courses. Education for
Chemical Engineers, 26, 80–88. https://doi.org/10.1016/j.ece.2018.11.006
Zar, J. H. (1984). Biostatistical analysis. (2nd ed.). New Jersey: Prentice-Hall, Inc.,
Eaglewood Cliffs.
Corresponding author: Lee Kok Yueh
Contact email: kokyueh.lee@utb.edu.bn

125

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Appendices
Appendix A: Coursework Handout
Scenario: You are an employee of Hengyi, and you are evaluating the viability of building an
oil refinery in Brunei (please ignore the fact that they are already building one). You have beenasked to compile a 600-word report outlining the following:
Introduction: Basic introduction to your report, which should include the following; descriptionof what an oil refinery does, including characteristic refinery size and life expectancy; main
feedstock; main products and market for those products; introduction to Brunei, it’s location
and resources. (100-150 words)
Discussion: This is where you discuss the advantages and disadvantages of locating a refinery
in Brunei. The points for discussion could include, but are not limited to; suitable geographicallocation; access to and quality of raw materials over life cycle of the plant; access to productmarkets over life cycle of the plant; possible issues which affect refinery profitability; humancapital resources; any social and political issues. (300-400 words)
Conclusion: This is where you highlight whether or not you think Hengyi should build arefinery in Brunei, along with the main arguments supporting your decision. (100-150 words)
Resources: You have been provided with a supporting document which you can use to startunderstanding where an oil refinery is placed within the “fuel supply chain”, along with someof the issues facing refineries. This document focuses on North America, and you are also
expected to undertake your own research when considering both refineries and the local
conditions present in Brunei.
Marking: This coursework will be peer evaluated by a fellow student in your class, and to help
guide them a marking rubrics will be provided by the lecturer. The evaluation is provisionallyscheduled to take place during a formally scheduled time slot in week 13 of the semester, prior
to the exams. The report will subsequently be second marked by the lecturer, before a final
mark is awarded to each student.

126

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Appendix B: Rubrics and additional information

127

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Appendix C: Student Questionnaire128

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

129

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

130

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Decision Making Model of Vietnamese Students Studying Higher Education in England
Minh Hoang
Saigon Institute of Technology, Vietnam
Massoud Moslehpour
Asia University, Taiwan
Victoria Seitz
California State University, San Bernardino, USA

131

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
Universities in England are now paying more attention to the Vietnamese student market for
purposes of campus diversity and recruitment. The primary purpose of this paper is to have abetter understanding of Vietnamese students’ motives and drives to study abroad. Morespecifically, this study targets students studying in a public research university in England. To
examine Vietnamese students’ decision-making processes for choosing a university, theresearch is conducted in two stages: focus groups and then structured interviews. Findings
reveal that the primary reason why Vietnamese students want to study abroad is because ofparents’ demands and English higher education qualifications. Secondly, it is because theywant to achieve better qualifications to improve job prospects. Students do not fully trust
agencies; however, they receive the most significant amount of information from agencies andsubsequently verify their information with friends, alumni, and other independent authorities.
Regarding choosing a university, the first and most influential factor is tuition fees and living
costs, followed by course modules. In general, students are satisfied with course modules andservices at the university. Although the findings are limited to the sample respondents that wereVietnamese students at this particular university, they are useful for university management
and international recruitment offices to improve services provided to students during therecruitment process. This study provides a better understanding of factors influencing
Vietnamese international students’ decision-making process.
Keywords: decision making process, international students, study abroad, Vietnamese, England132

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

British universities are falling behind international universities in the world. The number ofuniversities and colleges in England with a financial deficit has jumped by almost 70% in just
one year (Rhodes, 2019). In real terms, the teaching budget for 2019/2020 is 74% below the
2011/2012 budget (Bolton, 2019). This is raising fears that the British universities’ statusamong the global elite is decreasing (Busby, 2018). With the prospect of a bleak financial
outlook and the continued drop in ranking, universities and polytechnics in England are seeking
other sources of revenue such as international students. International students are defined as
students who sought higher education outside their home country and hold temporary student
visas (Andrade, 2006). International students in UK universities are charged higher tuition fees
than the local students; as a result, international students contribute a considerable amount ofmoney towards the funds (Binsardi & Ekwulugo, 2003) as well as contribute their talent andbring diversity to campuses (Rhodes, 2019). In 2017, international students contributed morethan £25 billion to the economy (Universities UK, 2017). Many other countries have also
undertaken various marketing and recruiting activities directed at international students. In
2016, there were over 4.8 million international students, up from 2 million in 2000 (Migration
Data Portal, 2017). More than half of these were enrolled in educational programs in six
countries: the US, the UK, Australia, France, Germany and the Russian Federation. The UK
accounted for around 10% of these international students (Study International News, 2018).
Vietnam is emerging as a country and worth paying attention to. The number of Vietnamesestudents who choose to study abroad has been increasing gradually year by year. In 2018,
approximately 130,000 Vietnamese students studied abroad and spent about USD 4 billion(Vnexpress, 2018). However, only 12,000 Vietnamese students were studying in the UK in2018 (Ha Phuong, 2018), accounting for only a small percentage of these Vietnamese overseas
students. The statistics suggest that the United Kingdom is not as attractive as other countries
for Vietnamese students. To attract and to market to Vietnamese students, it is necessary for
English universities to understand their perceptions and behaviors in order to attract them. The
purpose of this study is to investigate factors influencing university enrolment decision-making
processes of Vietnamese students. More specifically, this study will look at Vietnamesestudents at a public university in England. Privacy and discretion of participants are central to
ethical research practice in social research (Wiles, Crow, Heath, & Charles, 2008). This study
is a low risk study. However, every effort is made to ensure that the data cannot be traced backto the participants or “the university”. To preserve anonymity and confidentiality we use
pseudonyms for participants and also for the location of the research.
Literature Review
The decision to study abroad is complex and involves deep deliberation in all stages in thedecision-making process. The literature identifies five stages: problem recognition,
information search, alternative evaluation, purchase decision, and post-purchase behavior(Solomon, 2017). Most students have limited experience, if any, of overseas higher educational
opportunities, so they are involved in extensive information searching (Rafi, 2018). Studying
abroad also involves high costs such as tuition fees, living costs abroad, and reduction ofsavings (Cebolla-Boado, Hu, & Soysal, 2018), as well as cultural and physiological costs(Madden & McMillan, 2018). Hence, students extensively evaluate benefits they might gain
during and after studying abroad and subsequently worry about progress and performance inthe course (Petzold & Moog, 2018).

133

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Perceptions and behaviors of students are different from country to country. Hence, it is notpossible to apply one strategy to all markets (Ma & Garcia-Murillo, 2018). Although overseas
students from Asian countries share similarities such as sharing the Confucian heritage andmoving to another country with a different culture for study, it would be problematic if thesestudents are seen as homogeneous (Norazlyn, Sweeney, & Soutar, 2016). For instance,
Malaysian students are found to be more cost-sensitive than Chinese students and prefer tostudy in the UK, while the latter favored Australia (Norazlyn et al., 2016). For Singaporean
students, the Singaporean dollar stayed strong against the Australian dollar and British poundover the last few years. As a result, more and more of the island country’s citizens gained themeans to study abroad in the latter two countries (Study International News, 2017).
Problem Recognition and Information Search
When students near the end of high school, they wonder what they want to do with their lives:
continue their studies at a university, go to work, or travel. If other students talk about their
university plans, this may trigger studying further (Chen & Zimitat, 2006). Parents, especiallythose from a wealthy and highly-educated background, also play a crucial role in inspiring andsupporting students to study overseas (Hulstrand, 2017). Many studies also find that studying
abroad is believed to increase job prospects, provide a better opportunity to improve theirEnglish language skills, and enhance their understanding of different cultures and traditions(Moogan, Baron, & Harris, 1999). Liwinski (2019) finds that Polish students studying abroadearn, on average, 22 percent more in their first job than those who never studied in a foreigncountry. Despite the pressure from the government, higher education institutions in Vietnam
are unable to provide statistics of employment rates to ensure their graduates’ employability(Tong, 2019).
Students primarily receive their information through commercial channels. For instance,
recruitment agents can make students’ choices easier and the application procedures simpler
because they have a vast network to obtain information. However, students also worry that
recruitment agents may give false or biased information (Rafi, 2018). University rankings areunbiased reviews from experts to provide students with reliable and comparable information
about higher education institutions worldwide, helping them make well-informed choices(Hou, Morse, & Shao, 2012). Electronic sources of information such as forums, blogs, andsocial networks also play an essential role in the decision-making process of prospective Asianstudents (Zhu, 2019). Unlike British and European students, Asian students are rarely able toattend open days at universities before deciding which university to choose because ofgeographic distance, so they rely heavily on the Internet to receive direct information fromuniversities (Marsden, Ibanez-Tirado, McMurray, Yusopov, & Halliel, 2019; Wilkins &Huisman, 2015).
Alternative Evaluation, Purchase Decision, and Post-Purchase Behavior
Medina and Duffy (1998) suggest that evaluative criteria to choose a university involve five
primary dimensions: learning environment, reputation, graduate career prospects, destination
image, and cultural integration. Students choose English universities because of theirreputation; other reasons are the effective and efficient procedures of admissions andimmigration, and the opportunity of finding a job while studying in England as well as back intheir home countries (Sá & Sabzalieva, 2018). The influence of parents is especially true if
they are paying for their children’s education, and parents have an immense influence regarding
this decision (Hulstrand, 2017). Other factors influencing students’ choice of an English
university include visa requirements by the UK Border Agency (UKBA), English proficiency,134

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019and evidence of funding (Tannock, 2018). In terms of English proficiency, many empiricalstudies indicate that English proficiency is an important indicator of academic success forinternational students (Martirosyan, Hwang, & Wanjohi, 2015). English proficiency plays acrucial role for international students in completing their studies in English language as wellas adapting with different educational systems, teaching styles, and relationships with teachersand friends (Martirosyan et al., 2015). For example, those with weak English language skills
have poor academic performance because they do not understand well a course’s requirements
and lessons (Martirosyan et al., 2015). They are also more likely to be isolated from local
students and faculty members (Jehangir & Khan, 2015).
Students’ educational experiences with a university’s academic programs and augmented
services provided over the courses influence student satisfaction with the institution(Elsharnouby, 2016). Augmented services consist of administrative services, staff, physicalcharacteristics of academic facilities, and social environment and advising support (Parahoo,
Harvey, & Tamim, 2013). Among these augmented services, maintenance activities (i.e.,
housing, food, and clothing) and campus life (i.e., availability of social and recreational
activities, library services, quality of classroom instruction) significantly influenceinternational students’ satisfaction (Elsharnouby, 2016). According to Solomon (2017),
universities that score high in student satisfaction often have a competitive advantage throughan enhanced reputation that aids in the recruitment of additional students. Satisfied students
are more likely to communicate openly, make constructive and insightful suggestions to auniversity, maintain the relationships with their institutions after graduation, and be potential
donors in the role of alumni (Parahoo et al., 2013). In contrast, dissatisfied students tend toleave the institutions quietly or share their negative experiences with their circle of friends(Eisingerich, Auh & Merlo, 2013).
Research Method
This study includes two stages: focus groups (qualitative) followed by structured interview
surveys (quantitative). We followed the steps suggested by Mack, Woodsong, Macqueen,
Guest, & Namely (2005) for focus group qualitative study. In this study non-probability
sampling was used with Vietnamese students studying at a public research university in
England. Focus group interviews were conducted to have a better understanding of theparticipants’ decision-making processes. The focus group consisted of three males and four
females. All participants were obtaining master degrees. The focus group sessions wereconducted in English instead of Vietnamese and tape-recorded to avoid biases and inaccuracy.
The discussions during focus group sessions were subsequently transcribed verbatim,
summarized and grouped into five categories corresponding with the five stages of thedecision-making process. These transcripts were analyzed to discover factors and explore
themes, patterns, and relationships between factors to form the questionnaire for the structured
interviews. For ethical reasons, participants’ names and information are kept anonymous. The
names mentioned in this study are fictional.
The questionnaire consisted of two main parts. The first part included demographic questions:gender, age, hometown, courses, and financial sources. The second part included identified
variables from the focus groups arranged in the five stages of the decision-making process:problem recognition, information search, evaluation, purchase and post-purchase. Statements
were measured using multiple-choice, a five-point Likert scale, and ranking. Thirty-three135

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Vietnamese students agreed and completed the questionnaire in the structured interviews. SPSS
was used to analyze the data.
Findings
Qualitative Findings from the Focus Groups
Reasons to study abroad (Problem-Recognition Behavior). Participants note that theVietnamese education system cannot satisfy their demand in knowledge provision and that theteaching style is passive and boring. Studying abroad is believed to help them gain international
educational experience, working with lecturers, tutors, and international students from all over
the world. Hence, to them, the Vietnamese labor market seems to favor those with foreign
qualifications over those with local ones. Respondents’ reasons to study abroad aresummarized below:
•
•
•
•
•
•
•
•
•

•
•
•

“I don't like the Vietnamese teaching style because it lacks communication between
students and lecturers”.
“I think the assessment in Vietnamese universities is unfair because sometimes when I
took an exam, I knew about the questions in advance; teachers recycled questions fromthe previous years, so I can gain a very high mark without studying hard”.
“I would like to learn some practical things instead of theoretical knowledge as in the
Vietnamese education system”.
“Studying abroad is one of the requirements of my university. To teach myundergraduate students, I, have to get a master degree from a foreign university within
three years”.
“In Vietnam, we don't have two kinds of classes like here (in England): lectures andseminars. We just have lectures in which lecturers keep speaking, and students keep
listening”.
“Vietnam is a developing country and there are an increasing number of foreigncompanies opening branches in Vietnam. So, if I study abroad, I can have better English
and better qualifications to broaden my job opportunities in the future”.
“The Vietnamese government wants to improve educational quality, so they give a lotof scholarships for students to study abroad”.
“I think a lot of companies are willing to recruit students who studied abroad. So thesestudents will have more job opportunities with higher salaries than those students
studying domestically”.
“My father really wants me to study abroad to gain the new experience from otherforeigners such as living or working styles. He always asked me, "Why don't you study
abroad like your friends?" He is willing to support me financially, so I think there is no
reason for not going. But I chose where to go”.
“If around you, a lot of friends study abroad, you also want to study abroad. You also
want something similar to what your friends do. It gradually becomes a demand in yourmind””“If I can pay more than 700 million VND to study abroad, I think I do not need to worry
much about the living costs here. But I still want to have a part-time job because I wantto socialize with British people to learn more about British culture and life”.
“The Vietnamese education system is not as good as that in England. Not allinternational companies accept a Vietnamese degree, but with an English degree, we
can apply for any company in the world”.

136

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Information search about universities (Information Search Behavior). Focus group
participants collect information from five main sources: friends, agencies, website/Internet,
conferences, and university ranking tables. However, all participants consider informationfrom friends the most reliable source. Some participants think that agencies can help save timeand prepare profiles properly. Nevertheless, the participants do not fully trust the information
provided by agencies because they perceive the information to be distorted to benefit agencies
rather than students. Therefore, the participants double-check the information with alumni fromthe universities where they intend to go in order to compensate for this distortion. Respondents’answers to their source of information are summarized below:
•

•
•
•

•
•
•

•
•
•
•

“At that time, some colleagues of mine already studied at this university and told methat this university is very good. I trusted them. Then I went to the official website of
the university to search my course, read requirements, and apply for it. I did notcompare this university with others. To me, university ranking is not important because
I don’t view the university’s history important, but I care about its social life.”“I searched for information about education in England on the Internet, for example, insome forums”.
“I searched Google to find some agents who could assist me to prepare documents tostudy in England”.
“Moreover, I asked friends on Facebook who used to study in England regarding social
life, teaching methods, campuses… On Facebook, I searched the phrase "VietnameseStudent Union in Sheffield", "Vietnamese Student Union in Birmingham", or"Vietnamese Student Union in Manchester"…”.
“I also attended several educational conferences to get information and communicate
with alumni. Sometimes, I also submitted my CVs there, but I see that my profiles later
are handled by an agency”.
“After I finished my undergraduate degree, I didn't have much time to prepare myapplication profile and find information about universities in England”.
“I tried one agency and it persuades me to study at the London Met, but when I camehome and searched on the Internet, I received a lot of bad comments about thisuniversity. So, I thought this agency was not good. I did not choose the London Met
and also changed to another agency. And it was lucky for me when I did not choose it
because this university collapsed some months ago”.
“So, I chose an agency recommended by my cousin. This agency made everythingeasier for me because they knew almost everything in England. They were very
professional and had good relationships with universities in England”.
“I think the most important source is from my friends because they are my friends andbecause they are studying there. They know exactly what happens in England, so they
can tell me exactly about the social life or teaching styles are”.
“When I was in Vietnam, I had very little information about what exactly happens here
(in England). To me, ranking tables are one reliable source of information about how
good a university is”.
“It is quite convenient to have an agency’s assistance. They provided me withinformation about the English requirements of universities in England, living costs ineach city, especially a list of which universities were available for January intake
because I started studying in January”.

137

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Why do Vietnamese students choose England? (Alternative-Evaluation Behavior).
Compared to other countries such as the USA and Australia, England is a chosen destination
because of its reputation in education, shorter time for studying, lower living costs and tuition
fees, and lower English requirements. Other participants also note that they want to improvetheir English in England – the homeland of the English language and can travel to Europe
easily during the summer holiday. Respondents’ criteria to choose a university are:
•
•
•

•
•
•
•

“I like the structure of modules and the program or courses at this university comparedto others. The English requirement is also low, just IELTS 6.0”.
“I want to study in an English-speaking country because I can improve my English
much faster. It is good when this university accepts IELTS 6.0 but such low
requirements may affect the quality of learning and teaching”.
“I compare Australia to England. Both of them have a good reputation in education.
However, I choose England because of several advantages. First of all, studying
duration in England is less than in Australia. As you know, in Australia, one Mastercourse lasts for one and a half year, so if I learn in England, I can save my money, savetime. Second, the exchange rate from AUD to VND or from USD to VND increases;
whereas due to Brexit, the exchange rate from GDP to VND decreases, so the total costs
for studying in England seems cheaper than other countries”.
“The agency gave me a list of universities based on my scores, budget, and interest, but
I chose this university because my close friend is here. He had been studying in Englandfor 3 months before I came here. I think he could support me when I study here”.
“My father only wants me to study abroad at an affordable university with a good
reputation. Choosing which country or university is my own decision”.
“The first thing I am concerned with when I choose a university is whether it has anavailable course in January or not. I had to finish my work before I went to study, so I
missed the September intake but I did not want to wait till the next September”.
“I chose this university because my friends said that this university was more practicalthan some of the other universities. English requirements here are low also. It is goodat the beginning for enrolment, but then it may affect the quality of the programs”.

Satisfaction/dissatisfaction with the current choice (Post-Purchase Behavior). In general,
all participants are satisfied with this university. First, they enjoy the small class sizes with 25
or 30 students where they can interact more with fellow students and instructors; in Vietnam,classes have at least 60 or 70 students. Second, teaching methods at this university are more
active and practical than in Vietnam with two kinds of classes: lectures and seminars, while
Vietnam provides only lecture classes. Third, lecturers and staff are helpful, friendly,
enthusiastic, sympathetic toward international students’ English skills, and especially reply toinquiries promptly. Fourth, library, online databases (i.e. Library Gateway, Google Scholar,
ebooks, …), computers, or books at this university are considered to be better than in
Vietnamese universities. However, some participants note that they are dissatisfied with course
programs because the courses are not as practical as they thought and because there are alimited number of optional modules for January intakes compared to September intakes. In
addition, some participants complain that there are too many Chinese and Vietnamese students
rather than the diversity of students they were seeking. Satisfaction and dissatisfaction of theparticipants with this university can be summarized as:•“Small classes with 20-25 students”.

138

IAFOR Journal of Education

•

•

•

•

•
•
•
•
•

Volume 7 – Issue 2 – Winter 2019

“One thing that I'm not really satisfied with is the program. I studied Bachelor inInternational Hospitality and Tourism Management and expected that I could get more
real information, more knowledge, and more case studies at the Master course at thisuniversity; but when I came here, I just study, write, and read with all subjects which
are not really new. I study what I already studied in the bachelor degree program.
Nothing new”.
“I know that the environment with too many Vietnamese students hinders my ability
from speaking English. So when I study here, I want to choose a course which has only
one Vietnamese student – that’s me - in the class. I think I am happier than thosestudents whose courses have more than 10 Vietnamese students. I want to learn more
about foreign culture. It doesn't mean that Vietnamese students here are not good; it’s
just because I want to learn different things”.
“I think our module looks quite practical, but honestly, it is not that practical. Like myfriends, they study in Leicester; they have very good practical modules and long-term
projects for finance-and-banking students. At this university, projects are mainly formarketing students”.
“Our aim is to get practical knowledge to work after we finish the course, not to get ahigher degree like a PhD but we don't have enough practical knowledge. It (theknowledge taught at this university) is just based on the books and articles. They are
not practical things”.
“There are quite a lot of Asian people, especially Chinese in my class. They always
speak in Chinese. It's very difficult for us to communicate with them”.
“I think the best place at this university is assets center (library). I have spent quite a
lot of time here. I think the technicians and librarians support me quite well. The systemsometimes has some small technical problems, but I think it is alright”.
“There are too many Vietnamese students in my class. I feel that I am studying in a
Vietnamese university, not an English university.”“I'm January-intake student in Finance and Banking. I see that a September intake has
about five or six optional modules to choose, and those modules are very exciting, while
my course has only two optional modules”.
“I will consider a prospective student’s ability first before I recommend this university
to their friends and relatives. If he or she wants to gain practical knowledge and had an
average ability and budget, I am willing to recommend this university. However, if heor she has a higher ability to study, larger budget, or want to gain better academic
knowledge, I would recommend other universities with a higher rank in university
ranking tables”.

Quantitative Findings from the Structured Interviews
The sample size was 33 people with over 90% between the ages of 22 and 28, and two thirds(63.64%) females. Almost all (97%) were studying for their master’s with the remaining
studying other qualifications such as bachelor’s degree, PhD, or English certificates at thisuniversity. Approximately 85% of interviewees earned their last degree in Hanoi with theremaining from other provinces in Vietnam such as Ho Chi Minh City, Da Nang, or Can Tho,suggesting that the students in Hanoi seem to favor England more than students in otherprovinces. Nearly 80% were sponsored by their parents, indicating that financial support fromfamilies is very important. Finally, two-thirds of the respondents are studying in January intake.

139

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Problem recognition. The data, as shown in Table 1, indicate that parents trigger the demand
for studying abroad in students with 24.2% Strongly Agree and 60.6% Agree, while friends do
not really influence at this stage (48.5% Disagree and 12.1% Strongly Disagree). English
higher education qualifications are considered to be more valuable than Vietnamese
qualifications (94% Agree and Strongly Agree).
Table 1: Reasons for the decision to study abroad
Statement No.
1. The Vietnamese education systemcannot satisfy my demand in knowledge
provision.
2. My company requires me to have aforeign higher education degree.
3. English higher education qualifications
are more valuable than Vietnamese
qualifications.
4. A lot of my friends study abroad, so I
want to study abroad, too.
5. My parents always encourage me to
study further, especially studying abroad.
6. I come of age to live independently
from my parents and experience a new
culture.

n

SA
5

A
12

UN
9

D
7

SD
0

%

15.2

36.2

27.3

21.2

0

n
%
n

2
6.1
17

10
30.3
14

11
30.3
1

11
33.3
0

0
0
1

%

51.5

42.5

3.0

0

3.0

n
%
n
%
n

4
12.1
8
24.2
11

5
15.2
20
60.6
20

4
12.1
4
12.1
2

16
48.5
1
3.0
0

4
12.1
0
0
0

%

30.3

66.6

6.1

0

0

Information search. Friends, Website/Internet, and Agencies appear to be the three mostimportant sources with 81.8%, 75.8%, and 72.7% respectively. However, as demonstrated inTable 2, the difference between sources of information is minimal.
Table 2: Source of information for the decision to study abroad
Friends
Yes
No
Total

n
27
6
33

Agencies
%
81.8
18.2
100

n
24
9
33

%
72.7
27.3
100

Website/Internet
n
%
25
75.8
8
24.2
33
100

Conferences
n
17
16
33

%
51.5
48.5
100

University
Ranking
n
%
22
66.7
11
33.3
33
100

Table 3 shows that Tuition Fees/ Living Costs, Course Modules/ Programs, and Facilities are
considered to be the most important criteria for the students seeking to study (84.8%, 87.9%
and 87.9% respectively).

140

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 3: Criteria for the decision to choose university abroad
Tuitionfees &
livingcosts

Scholarship/ Teaching
financialmethods
aids

Coursemodules/
programs

Lecturers& staff

University
reputation

Facilities

n

%

n

%

n

%

n

%

n

%

n

%

n

%

Yes
28
No
5
Total 33

84.8
15.2
100

22
11
33

66.7
33.3
100

25
8
33

75.8
24.2
100

29
4
33

87.9
12.1
100

21
12
33

63.6
36.4
100

21
12
33

63.6
36.4
100

29
4
33

87.9
12.1
100

Alternative evaluation. The most influential factor for choosing a university in UK is theshorter duration of the master’s degree programs compared to other countries (81.8% Strongly
Agree and Agree), but tuition fees and living costs are not (48.5% Disagree and Strongly
Disagree). This result agrees with the findings in the focus groups that finds England is chosenbecause of the duration of courses in England is less than that in other countries. Most of theparticipants agree that they choose England because of job prospects, English improvements,
cultural experience, English requirements and travelling as shown in Table 4.
Table 4: Reasons for the decision to choose an English university
Statement No.
9. I want to improve my job prospects
10. English qualifications are recognized over
the world
11. The duration for higher education programs
in England is less than in other countries.
12. Tuition fees and living costs in England areless than in other countries
13. I want to improve English in the homelandof this language
14. I want to experience Western culture
15. English higher education requires only
IELTS whereas other countries such as the
USA require more certificates such as TOEFL,
GMAT, or GREE
16. I want to travel in Europe while studying in
England

n
%
n
%
n
%
n
%
n
%
n
%
n

SA
20
60.6
12
36.4
13
39.4
1
3.0
16
48.5
9
27.3
3

A
13
39.4
19
57.6
14
42.4
8
24.2
12
36.4
17
51.5
13

UN
0
0
2
6.1
4
12.1
8
24.2
5
15.2
6
18.2
10

D
0
0
0
0
2
6.1
13
39.4
0
0
1
3.0
6

SD
0
0
0
0
0
3.0
3
9.1
0
0
0
0
1

%

9.1

39.4

30.3

18.2

3.0

n
%

7
21.2

18
54.5

7
21.2

1
3.0

0
0

The most important factor to choose this university is low tuition fees and living costs (87.9%),
followed by interesting course modules (75.8%). The influence of friends is the least importantfactor (18.2%).

141

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 5: Factors influencing students to choose this English university
January
intakeInterestingcoursemodules

Practical
teaching
methods

Tuitionfees &
livingcosts

Requirement Scholarship My
in IELTS& financial friends
aids
studying
here

n

%

n

%

n

%

n

%

n

%

n

%

n

%

Yes
19
No
14
Total 33

57.6
42.4
100

25
8
33

75.8
24.2
100

19
14
33

57.6
42.4
100

29
4
33

87.9
12.1
100

16
17
33

48.5
51.5
100

14
19
33

42.4
57.6
100

6
27
33

18.2
81.8
100

Purchase decision. This part examines who influenced students’ decision to choose this public
university. The data shown in Table 6 indicate that parents, friends and alumni are mostinfluential with 48.50%, 54.50% and 33.30%, respectively.
Table 6: Individuals influencing a student to make this decision

Yes
No
Total

My parentsMy
friends

My other family
members such as
brothers, sisters,
or cousins

My teachers

My boss

Alumni

n

%

n

%

n

%

n

%

n

%

n

%

16
17
33

48.5
51.5
100

18 54.5
15 45.5
33 100

8
25
33

24.2
75.8
100

9
24
33

27.3
72.7
100

4
29
33

12.1
87.9
100

11
22
33

33.3
66.7
100

Post-Purchase Behavior. The findings shown in Table 7 indicate that Vietnamese participants
do not like too many Chinese students in the class (78.8% Strongly Agree and agree). They
also report that they want to have more Western students (45.5% Strongly Agree and 42.4%
Agree). In general, this university has received very good feedback from participants, including
suitable course (84.9% for Strongly Agree and Agree), helpful and friendly staff and lecturers(84.9% for Strongly Agree and Agree), good facilities (78.8% Strongly Agree and Agree),peaceful campus and nice local people (72.7% Strongly Agree and Agree).
Table 7. Students’ satisfaction with their choice of this university
Statement No.
19. The course program is very
suitable for my ability
20. Staff and lecturers are very helpful
and friendly
21. Facilities such as the library,computers, or books are very good
22. This city is peaceful and local
people are very nice and friendly.
23. The course is not as practical as I
expected

n
%
n
%
n
%
n
%
n
%

SA
5
15.2
6
18.2
10
30.3
7
21.2
2
6.1

A
23
69.7
22
66.7
16
48.5
17
51.5
8
24.2

UN
3
9.1
4
12.1
6
18.2
5
15.2
10
30.3

D
1
3.0
1
3.0
1
3.0
2
6.1
11
33.3

SD
1
3.0
0
0
0
0
2
6.1
2
6.1

n

8

11

11

3

0

142

IAFOR Journal of Education

24. There is a limited number of
optional modules for January-intake
students (If you are a January-intake
student)
25. There are too many Chinese andVietnamese classmates
26. I wish my class would have more
Western students

Volume 7 – Issue 2 – Winter 2019

%

24.2

33.3

33.3

9.1

0

n
%
n
%

16
48.5
15
45.5

10
30.3
14
42.4

3
9.1
4
12.1

3
9.1
0
0

1
3.0
0
0

Discussion
Problem Recognition
Results from the focus groups and structured interviews reveal two important interdependentdetails. First, the Vietnamese educational system, employers’ requirements, and preferences
toward English qualifications over local qualifications trigger Vietnamese students to study
abroad. Secondly, Vietnamese students believe that English qualifications can improve theirjob prospects. English language skills provide opportunities for exchange of cultures as wellas travel while studying. These findings point to the fact that Vietnamese students preferoverseas education qualifications to local qualifications.
Nearly 80% of the participants studying at this university are sponsored by their parents.
Results from the quantitative data indicate that parents strongly influence students to study
abroad, but friends do not. Findings fully support the research of Hulstrand (2017) and Wadhwa(2016) regarding the role of parents, but challenge the research of Chen & Zimitat (2006) interms of the role of friends. Hence, it is important to inform and communicate with Vietnamesestudents’ parents as well.
Information Search
Results show that Vietnamese students seek information from many sources such as friends,
agencies, website/Internet, conferences, and university ranking tables. The participants receive
the largest amount of needed information from agencies, but check again with their friends andalumni from the universities where they want to study. Prospective students trust that the
information coming from personal sources (friends, family) or public sources (student
associations, university rankings) are independent and unbiased. This supports the findings ofRafi (2018) and Wadhwa (2016). They argue that friends and relatives are the primary sourcesof information used by prospective students followed by institutional coaching centers, theInternet, agents and education fairs. Hence, the university should have a close relationship withagencies and should build a strong alumni network for their significance to gain the trust ofprospective students and support information from agencies.
Alternative Evaluation
Focus group results (stage 1) indicate several factors influencing the choice of England overother countries. These factors include worldwide-recognized qualifications, shorter course
duration, lower tuition fees and living costs, the homeland of the English language, ease of
application procedures, and ease of travelling around Europe. In addition, some participants
mentioned Brexit, which caused the low exchange rate from GDP to VND, benefiting
Vietnamese students to study and spend in England. Except for low tuition fees and living costs,
results agree with the finding from the structured interviews. Hence, universities should

143

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019carefully review tuition fees and living costs in England compared with those in the USA andAustralia to attract international students.
Reasons to choose this public research university over others in England are low tuition fees
and living costs, practical teaching methods, January intakes, lower English requirements, alarger number of scholarships and financial aid, and friends currently studying at thisuniversity. From the structured interview results, low tuition fees/living costs are the two mostimportant factors to choose this university. This suggests the university appeals more toVietnamese students who are cost-sensitive. Respondents recommend this university to those
prospective students who have average ability and limited budget. These findings are consistent
with Wadhwa (2016) who found that tuition and institution fees, availability of scholarships
and financial aid, and quality of faculty influenced students to choose a particular hostinstitution. Lee (2014) also found that cost issues most influenced international students’decision-making processes whereas recommendation from others was the least important. Thissuggests that the affordability of the university appeals to the Vietnamese students who are
cost-sensitive.
Post-Purchase Behaviour
The participants consider the quality of course modules much more critical than university
services, given that the degree quality would positively affect their job prospects. However,they find it hard to evaluate the degree quality because they are still studying at this university
and job prospects are in the future. Therefore, participants tend to examine their satisfaction ordissatisfaction based on the immediate environment or factors such as courses, lecturers,
facilities, and classmates. This finding supports the outcomes of the Medina and Duffy (1998)
study. In general, participants are satisfied with the university’s course modules, library, local
people and a peaceful city.
Lower English requirements are one of the reasons that participants choose this public research
university, but such low standards may restrict Vietnamese students from participating in classand socializing with local people, leading them to isolation or mental problems. This finding
is fully consistent with Martirosyan et al. (2015) and Jehangir and Khan (2015). In the longrun, this issue may erode perceived satisfaction among students. Vietnamese students want toassociate with more Western rather than Asian students to improve their English and cultural
exchange; however, their courses are dominated by Chinese and Vietnamese students.
Conclusion
English university qualifications are perceived as more valuable than Vietnamese university
qualifications. Also, the Vietnamese Dong has stayed strong against the British pound over thelast few years, which makes British universities more affordable to Vietnamese students.
English universities should take advantage of this point to promote higher education programs
to Vietnamese students. English universities can not only attract students to England, but also
export the programs to Vietnam through dual-degree programs with Vietnamese universities.
Such a strategy would help some Vietnamese students solve some problems related to visa
requirements, English proficiency and evidence of funding as requested by UK Border Agency.
These dual-degree programs may also avoid brain drain and immigration issues in England
after students’ graduation.

144

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

As parents are the major sponsors for Vietnamese students to study abroad, marketingstrategies should address both students and parents. The information provided by friends,
alumni, or independent authorities such as university ranking establishments is more credible
than commercial agencies. Therefore, English universities should build close relationships withalumni networks and build reputation and recognition from trusted independent authorities.
The two most important factors in choosing this university are low tuition fees and living costs.
This university seems to attract students who are cost-sensitive with average ability. To attract
more talented students, the university should provide more financial aid to students.
Regarding post-purchase behavior, the core benefit of English qualifications is job prospects.
However, students find it hard to determine that while they are still studying. Hence, students
tend to evaluate their satisfaction or dissatisfaction based on the surrounding environment.
Students’ observations include: course modules, financial, employment opportunities while inschool and after graduation. Furthermore, students rely on well-informed advisors to help themadjust to course modules, the knowledge conveyed, teaching methods, and classmate diversity.
In general, Vietnamese students are satisfied with course modules and services at thisuniversity. However, some students are worried about low requirements for English capacity
for international students that may negatively impact their learning quality in higher education.
They are also worried about social isolation from local students. Thus, pairing internationaland host students may assist international students to adapt to the new culture and learning
methods better. Moreover, Vietnamese students have a desire to associate with Western
students rather than Asian students (i.e., Chinese or other Vietnamese students) to improvetheir English and cultural exchange. Given this, universities may wish to implement adiversification strategy in student recruitment to provide a diverse student body of domestic
and international students.

145

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
Andrade, M. S. (2006). International students in English-speaking universities: Adjustment
factors. Journal of Research in International Education, 5(2), 131–154.
https//:doi.org/10.1177/1475240906065589
Binsardi, A. and Ekwulugo, F. (2003). International marketing of British education: Researchon the students' perception and the UK market penetration. Marketing Intelligenceand Planning, 21(5), 318–327. https://doi.org/10.1108/02634500310490265
Bolton, P. (2019). Higher education funding in England, The House of Commons Library
Briefing Paper, Number 7393.
https://researchbriefings.parliament.uk/ResearchBriefing/Summary/CBP-7973
Busby, E. (2018). UK universities drop in global reputation rankings for research andteaching, Independent. Retrieved from https://www.independent.co.uk
Cebolla-Boado, H., Hu, Y. & Soysal, Y. N. (2018). Why study abroad? Sorting of Chinese
students across British universities. British Journal of Sociology of
Education, 39(3), 365–380. https://doi.org/10.1080/01425692.2017.1349649
Chen, C. H., & Zimitat, C. (2006). Understanding Taiwanese students’ decision-making
factors regarding Australian international higher education. International Journal of
Educational Management, 20(2), 91–100.
https://doi.org/10.1108/09513540610646082
Eisingerich, A. B., Auh, S., & Merlo, O. (2013). Acta non verba? The role of customer
participation and word of mouth in the relationship between service firms’ customer
satisfaction and sales performance. Journal of Service Research, 17(1), 40–53.
https://doi.org/10.1177/1094670513490836.
Elsharnouby, H. T. (2016). Participation behavior among international students: The role ofsatisfaction with service augmentation and brand choice attainment. International
Journal of Educational Management, 30(5), 679–697.
https://doi.org/10.1108/IJEM-10-2014-0139
Ha Phuong. (2018). Vietnam has 12,000 students in the UK. HanoiTimes. Retrieved fromhttp://www.hanoitimes.vn
Hou, A. Y.; Morse, R., & Shao, Y. E. (2012). Is there a gap between students' preference anduniversity presidents' concern over college ranking indicators? A case study of"College Navigator in Taiwan". Higher Education, 64(6), 767–787.
https://doi.org/10.1007/s10734-012-9524-5
Hulstrand, J. (2017). Working with parents: Strategies for success. International Educator,
26(3), 60, 62–63.
Jehangir, M. & Khan, R. (2015). English language proficiency: Social and academic
adjustment of Pakistani postgraduate students abroad. Journal of Humanities andSocial Sciences, 23(2), 73–82.
Liwinski, J. (2019). Does it pay to study abroad? Evidence from Poland. International
Journal of Manpower, 40(3), 525–555. https://doi.org/10.1108/IJM-11-2017-0305
Ma, Y. & Garcia-Murillo, M.A. (Eds.). (2018). Understanding international students from
Asia in American universities. Switzerland: Springer International Publishing.

146

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Mack, N., Woodsong, C., Macqueen, K. M., Guest, G., & Namely, E. (2005). Qualitative
research methods: A data collector’s field guide. California: Family HealthInternational.
Madden, T. M., McMillan, A., & Madden, L. T. (2018). This is not a vacation: The shadowside of study abroad programs for faculty. Journal of Management Education, 43(2),185–199. https://doi.org/10.1177/1052562918815212
Marsden, M., Ibanez-Tirado, D., McMurray, J., Yusupov, D., & Halliel, S. (2019).
Commissioned report for the Eurasian Council on Foreign Affairs – everyday
diplomats: new perspectives on networks, commerce, and regional integration inCentral Asia. Technical Report. Eurasian Council of Foreign Affairs.
http://www.eurasiancouncilforeignaffairs.eu/wp-content/uploads/2018/11/2.Eurasian-Council-on-Foreign-Affairs-July-2018.pdf
Martirosyan, N. M., Hwang, E., & Wanjohi, R. (2015). Impact of English proficiency on
academic performance of international students. Journal of International
Students, 5(1), 60–71. https://doi.org/10.32674/jis.v5i1
Medina, J. F. & Duffy, M. F. (1998). Standardisation vs globalisation: A new perspective ofbrand strategies. Journal of Product & Brand Management, 7(3), 173–178.
https://doi.org/10.1108/10610429810222859
Migration Data Portal (2017). International students. Retrieved fromhttps://migrationdataportal.org/themes/international-students
Moogan, Y. J., Baron, S. & Harris, K. (1999). Decision-making behavior of potential higher
education students. Higher Education Quarterly, 53(3), 211–228.
https://doi.org/10.1111/1468-2273.00127
Norazlyn, K. B, Sweeney, J. C., & Soutar, G. N. (2016). International students’ university
preferences: how different are Malaysian and Chinese students? The International
Journal of Educational Management, 30(2), 197–210.
https://doi.org/10.1108/IJEM-08-2014-0122
Parahoo, S., Harvey, H. & Tamim, R. (2013). Factors influencing student satisfaction in
universities in the Gulf region: does gender of students matter?, Journal of Marketingfor Higher Education, 23(2), 135–154.
https://doi.org/10.1080/08841241.2013.860940
Petzold, K., & Moog, P. (2018). What shapes the intention to study abroad? An experimental
approach. Higher Education, 75(1), 35–54.
https://doi.org/10.1007/s10734-017-0119-z
Rafi, M. (2018). Influential factors in the college decision-making process for Chinese
students studying in the U.S. Journal of International Students, 8(4), 1681–1693.
http://doi.org/10.5281/zenodo.1468068
Rhodes, D. (2019). Number of English universities in financial deficit increases. BBC News,
Retrieved from https://www.bbc.com
Sá, C. M., & Sabzalieva, E. (2018). The politics of the great brain race: public policy andinternational student recruitment in Australia, Canada, England and the USA. Higher
Education, 75(1), 231–253. https://doi.org/10.1007/s10734-017-0133-1

147

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Solomon, M. R. (2017). Consumer behavior: Buying, having, and being. New Jersey:
Pearson Education.
Study International News. (2017). More Singaporeans head to Australia, UK for studies as
currency stays strong. Retrieved from https://www.studyinternational.com
Study International News. (2018). Which country is home to the largest international student
population?. Retrieved from studyinternational.com
Tong, L. (2019). Graduate employment in Vietnam. International Higher Euducation, 97,
23–24. https://doi.org/10.6017/ihe.2019.97.10948
Vnexpress. (2018). Vietnamese families splurge $4 billion per year on education abroad.
Retrieved from https://e.vnexpress.net/
Tannock S. (2018). International students in the UK: Caught between market forces andimmigration targets. In S. Tannock, Educational equality and international students,(pp. 39–67). Cham, Switzerland: Palgrave Macmillan..
Universities UK (2017). International students now worth £25 billion to UK economy – New
research. Retrieved from https://www.universitiesuk.ac.uk
Wadhwa, R. (2016). Students on the move: Understanding decision-making process anddestination choice of Indian students. Higher Education for the Future, 3(1), 54–75.
https://doi.org/10.1177/2347631115610221
Wilkins, S., & Huisman, J. (2015). Factors affecting university image formation amongprospective higher education students: the case of international branch campuses.
Studies in Higher Education, 40(7), 1256–1272.
https://doi.org/10.1080/03075079.2014.881347
Wiles, R., Crow, G., Heath, S., & Charles, V. (2008). The management of confidentiality andanonymity in social research. International Journal of Social Research Methodology,
11(5), 417–428. https://doi.org/10.1080/13645570701622231
Zhu, Y. (2019). Social media engagement and Chinese international student recruitment:
Understanding how UK HEIs use Weibo and WeChat. Journal of Marketing forHigher Education, 29(2), 173–190. https://doi.org/10.1080/08841241.2019.1633003
Corresponding author: Massoud Moslehpour
Contact email: writetodrm@gmail.com

148

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Academic Staff Induction and Assessment on Implementing Experiential Learning andReflective PracticeVera Maria Nistor
Otago Polytechnic, Auckland International Campus
New Zealand
Don Amila Sajeevan Samarasinghe
Otago Polytechnic, Auckland International Campus
New Zealand149

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
Experiential learning and reflective practice are two effective key learning and teachingstrategies that many successful teachers employ as learner-centred education practices. It has
been proven that many students appreciate the meaningful learning received through a learnercentred classroom environment.
The aim of this paper is to share the findings of an induction programme focussing on
experiential learning and reflective practice, which was designed to enhance the importance oftransforming teacher-centred practice into learner-centred practice, with the teacher taking on
the role of facilitator. The objectives of this induction programme were to help staff inunderstanding and incorporating experiential learning models in their teaching practice at a
New Zealand tertiary institution.
The paper contains a qualitative analysis of staff perspectives on experiential learning practicetaken from their answers to a questionnaire, which followed an online Moodle course. It also
contains a quantitative analysis of the data gathered from classroom observations using a digitalclassroom observation tool for assessment.
The results showed that, although theoretical awareness about experiential learning was not a
problem for new academic staff, its practical application was not always fully achieved. Even
when lecturers were successfully implementing experiential learning activities in their classes,
they were not incorporating reflective practice as part of a complete process. A more conscious
incorporation of reflective practice by the academic staff in their classes, will ultimately help
students to continue learning through experiences and reflection in their self-directed learningjourneys.
Keywords: experiential learning, reflective practice, staff induction programme, teacher
performance150

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Experiential learning and reflective practice are the key drivers for bringing an authentic andlearner-centred environment into classroom practices (Beard & Wilson, 2015; Moon, 2004).
This is probably why experiential learning is part of many tertiary institutions’ curriculum
requirements nowadays, as a preferred learning and teaching method.
The basis for experiential learning is the constructivist belief (Vygotsky, 1978) that students
learn best by trialling and experimenting with what they are taught, rather than by passively
observing or listening to the teacher. Many teaching practitioners seem to understandexperiential learning as “learning through reflection on doing” (Felicia, 2011, p. 1003). Yet, inorder for the students to obtain a complete learning experience using this method, theirexperiential learning tasks need to be accompanied by a reflective session (Moon, 2004).
In modern teaching practice, experiential learning involves problem-solving learning
experiences (Miller & Maellaro, 2016) and co-created learning experiences that enhance
student capabilities and transferable skills (Stephenson, 1998). This would ideally enable
students to understand how they learn and how to continue to learn, and hopefully also
positively contribute to their future employability.
This paper aims to share the findings of a trial induction programme designed to assess theextent to which academic staff at a New Zealand tertiary institution are employing experiential
learning activities within their regular classes. The main objective of the induction programme
was to support new academic staff members at our New Zealand tertiary institution andenhance their understanding of experiential learning models, incorporation of experiential
learning and reflective practice into the courses, and contextualising experiential learning
within the institution’s learner capabilities framework. This framework was designed to offer
students the necessary knowledge and skills to become successfully employed in their field in
the future. The trial induction programme was run by the authors as peers within our institution,
coming from different teaching backgrounds and subjects (the experiential learning team).
Based on a continuing trend for tertiary educators to deliver their classes in a teacher-centred
way, this trial induction programme was designed to try and change their perception and toshow (mainly through professional conversations with the experiential learning team) thatstudents would get better learning opportunities through a more practical, experiential learning
strategy.
The paper explores both teaching practitioners’ reflection on what experiential learning means
for them, as well as the practical task of applying and incorporating this model into their daily
teaching practice. Conclusions are drawn from the findings on both aspects, and somerecommendations for the future are made on how a more comprehensive experiential learning
curriculum could be further achieved.
Literature Review
Experiential Learning
In education, Kolb’s learning cycle of experiential learning (1984) is one of the most wellknown experiential learning models. According to this model (based on Lewin’s 1951
experiential learning model), experiential learning follows a continuous learning cycle which
revolves around having concrete experiences (doing), reflecting on these experiences(reflective observation), followed by processing this information and understanding linkages(abstract conceptualisation), and eventually planning and trying out what has been learnt within
other contexts (active experimentation). Summarising Dewey’s classic model of the learning151

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

process, Kolb (2015) concluded that experiential learning is a type of learning that “transforms
the impulses, feelings, and desires of concrete experiences into higher-order purposeful action”(p. 33), which shows the transferability of these experiences. Not unlike the abstract
conceptualisation stage in Kolb’s model, Boud’s model of learning from experience alsostresses the fact that students come with existing knowledge or a personal foundation ofexperience (Boud, 1994) and all new knowledge needs to be assimilated through this alreadyexisting knowledge, in order for successful acquisition of knowledge to occur. Prior learningand new learning are grounded in experience and reflective practice. An important idea toconsider here is that learning is a personal matter and not every student learns the same way(Beard & Wilson, 2015). This idea supports Gardner’s well-known theory of multiple
intelligences (1999), which argues that different students can prefer to learn in seven different
ways (such as linguistic, logical-mathematical and bodily-kinaesthetic), depending on their
type of intelligence.
Experiential learning activities are commonly classified into two different types: field-basedlearning and classroom-based learning (Lewis & Williams, 1994). This stresses the practicalityof this model of learning, and shows how experiential learning does not need to stop within a
traditional classroom-based educational model. Beard and Wilson (2015) list a number ofdifferent types of field-based activities that could be experiential, such as adventure learning
through outdoor team activities, dramaturgy and field trips. Experiential learning comes withinteractivity and participation, and is considered more effective than traditional learning
approaches.
Reflection in Experiential Learning
In spite of its growth in popularity, there has been criticism of some teaching practitioners
misunderstanding the real meaning of experiential learning. Beard and Wilson point out therisk of an activity trap, where teachers use a multitude of experiential learning tasks in theirclasses, but these turn out to be uncritical activities, devoid of any reflective component (2015,
p.124). Ultimately, without reflection, students might not even end up learning from doing
experiential tasks in the classroom.
Reflection is a way of learning through thinking. Dewey (1938, p. 9) defined reflection as “theactive, persistent, and careful consideration of any belief or supposed form of knowledge in thelight of the grounds that support it and the further conclusions to which it tends”. There are two
major ways of reflection: reflection in action and reflection on action (Schon, 2017). Reflectionin action refers to the sequence of thoughts we have while doing an action. Reflection on actionrefers to the reconstruction process of the action once it has been concluded, to draw lessons
from the experience (Brookfield, 2017). What both of these types of reflection have in common
is the processing of information based on an event that is occurring or has occurred, in order tolearn from it. It is what Moon calls “cognitive housekeeping” or “sorting out of bits ofknowledge, ideas, feelings, awareness of how you are behaving and so on”. (Moon, 2004,
p.187).
Depending on its depth, reflection can be categorised into descriptive writing, descriptivereflection, dialogic reflection and critical reflection. Descriptive writing refers to shallow
descriptions that do not include a discussion about what happened. Therefore, it does not
involve reflection. In descriptive reflection, there is a description of events. However, only one
perspective is acknowledged. Dialogic reflection analyses multiple perspectives outside theevents discussed, demonstrating an in-depth reflective result. In critical reflection, actions andevents are deeply analysed by multiple perspectives located in and influenced by historical and152

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019social-political contexts (Moon, 2004). Critical reflection would therefore help to create moredemocratic and inclusive classroom environments (Brookfield, 1995). Moon (2004) argued
that reflection could deliver unexpected outcomes (ideas that could have been solutions todilemmas, creative activities), emotions and recognise the need for further reflection.
Therefore, reflection is not only part of a good quality learning process but also important inlearning behaviour development (Brookfield, 1995).
It has been noticed that, even though reflection occurs as a natural phenomenon, it is rarely
brought up for critical discussions with learners (Cavilla, 2017). Therefore, lecturers mightneed to review their teaching practice and improve the reflection activities they use in theirclasses. For instance, it would be good practice to include weekly reflective journals in any
type of class, irrespective of the subject taught. Forums, and question and answer sessions, canalso be set up in which learners can share their ideas on reflective practices. Boud (2010) argued
that when reflecting, the emotions that are associated with the event must be considered (suchas fear, panic hesitation, calm or elation). Emotion plays a vital role in quality reflection, as
feelings are part of any genuine reflection (Boud, 2010; Pedler, 2011). Therefore, it issuggested to slow down the pace of learning and encourage more emotion to make meaningout of experiences (Moon, 2004). Engaging emotions in reflective sessions would enable bothstudents and lecturers to validate the findings from reflective activities. Eventually, bothstudents and lectures would engage in reflective practices naturally. Ideally, reflection should
happen regularly as part of everyday learning (Moon, 2004).
Post-Observation Staff Reflection
The values and beliefs of a lecturer determine their philosophy (Schonell et al., 2016). Post
observation reflective sessions could help lecturers to discuss and understand what is essential
in the delivery of quality education. Upon understanding what values lecturers hold, they canthen develop good relationships with their students, which would grow over time and which
would result in an environment conducive to learning (Hamre & Pianta, 2006). The post
observation reflective session would ideally also encourage lecturers to reassess their passion
for teaching. Similarly, Rashidi and Moghadam (2014) argued that there is a positive
correlation between a teacher’s sense of complacency and poor student achievement. In orderto keep themselves up-to-date through professional development activities, lecturers could
benefit from being observed by peers, such as learning and teaching teams, who can provide
them with constructive feedback (Ahmed, Nordin, Shah, & Channa, 2018).
Experiential Learning Staff Induction Process
The induction programme was designed in four main stages. It was aligned with the staff peer
observation model proposed by Murphy Tighe and Bradshaw (2013) which included
preparation, pre-observation meeting, observation and post-observation feedback session.
In the first stage, staff were introduced to experiential learning and reflective practice throughan online activity (a short Moodle course with videos and other reading materials). The key
objectives of this course were:•••

To help the staff to understand what experiential learning means;
To support the staff in developing ways to incorporate experiential learning andreflective practice into their courses;
To encourage the staff to contextualise experiential learning within the capabilities’
framework.

153

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

This was intended to have them experience the process by themselves. Following the online
course, the participants had to complete an online questionnaire (eight questions), testing theirgeneral knowledge on experiential learning. An automatic email was generated to theexperiential learning team when the questionnaire was submitted.
The second stage of the induction programme was a face-to-face reflective session with theexperiential learning team, to clarify any specific issues the staff might have had on experiential
learning practice following the online course and questionnaire. Staff were then asked to comeup with potential ways of incorporating experiential learning activities into their respective
courses.
In the third stage, a class observation, using a digital observation tool, completed the induction
programme. They were primarily observed on their application of experiential learning
activities. The observation tool captured the learning environment in terms of the different
types of learner-centred (including experiential learning and reflection) or teacher-centred
activities and the positions of the classroom facilitator. It also provided a map of teachers’
movements in the classroom every 30 seconds.
The fourth and final stage of the induction programme was a face-to-face reflective session
with the experiential learning team to discuss feedback from the class observations. Thissession was designed to reflect on the class observation results. It provided opportunities forboth observer and teacher to discuss their approach to experiential learning and reflective
practice and create a development plan for future improvement.
Research Methods
The staff induction programme was initiated by and conducted with the approval of theacademic board of our institution. The research participants were informed in advance about
the purpose of the staff induction programme, how it was going to be conducted and what was
expected of them. The primary data (from the online questionnaire and class observations)were collected in a responsible and ethical manner by using codes instead of participants’names, thus ensuring that their anonymity and confidentiality remained unaffected at all times.
There were 16 new staff members who went through our experiential learning induction
process. Our findings from the induction process included both qualitative and quantitative
data. A mixed-method analysis was employed to analyse both qualitative and quantitative
research findings as it helped to optimise the benefits of both methods (Morgan, 2007). In otherwords, we increased the validity of our research findings by triangulating qualitative datacollected from the online course with quantitative data collected from the class observation
results (Bryman, 2016).
The qualitative information received from the online course was analysed using content
analysis in order to discover what academic staff knew about experiential learning andreflective practice in theory. The responses received for the eight online questions weretabulated. They were then summarised based on the themes that emerged. It should be notedthat one participant might have provided more than one response to a question asked.
Therefore, in each question, the popularity of a theme was calculated as a percentage of thetotal number of responses received. The themes were then ranked based on their popularity toidentify the significant themes generated from the online questions. The themes generated areshown in Figures 1-8.

154

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

The quantitative data captured by the digital observation tool was statistically analysed to
explore the extent to which academic staff applied their theoretical knowledge about
experiential learning and reflective practice to their classes. This was done by identifying thefrequency of teacher-centred and learner-centred class activities demonstrated during classobservations (as shown in Figure 9). The legend for these different types of activities
incorporated within the observation tool is shown in Figure 10.
Results
Qualitative Data Analysis and Findings
This section presents the results generated from the qualitative analysis of the responses
received from the first stage of the induction programme. As mentioned above, there were eightquestions related to experiential learning and reflective practice. The responses are shown inorder of popularity (from the most popular to the least popular). The responses with less than
5% popularity were eliminated.
Question 1: What is experiential learning and why is it important?

Figure 1: The definition of experiential learning and its importance
The most popular response (24%) was that experiential learning suggests a hands-on approach,in which students learn by doing, as well as through reflection (17%) and having relevant
experiences in the classroom (20%). One participant said:“As a natural model it mimics what happens to us unconsciously anyway andmakes that process conscious to us”.
Many participants also believed that the concept suggests a spiral of continued lifelong learning(10% of responses - some alluding to Kolb’s experiential learning cycle), and that students are
generally more motivated to learn in this way (9%). Some also believed that experiential
learning makes students become more autonomous (6%) and enhance their capabilities andtransferable skills (6%). Another participant added:“Experiential learning provides learners access to each stage of problem-solving,and extends the learning experience from limited school life to life-long period”.

155

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Question 2: What makes the experiential approach different from conventional
approaches to tertiary education?

Figure 2: The key characteristics of experiential learning
A high number of responses (17%) stated that the experiential approach is more student-driven
than conventional approaches to tertiary education. 14% stated that it is more practice-centred
and relies on active participation.
A further 11% of responses showed that the experiential approach equips students more for thereal world than a conventional approach, as it is always based on a real-world and problemsolving context. One participant said:“Conventional tertiary teaching practice views students as empty containers or
filing cabinets that need to be filled. It is teacher-driven didactic learning.
Experiential learning views students as knowledge generators; as capable
independent learners able to implement ideas and learning in a range of contexts.
It is student-driven”.
Many responses (10%) also indicated that, with an experiential approach, the classroom is used
as a space for reflection. 8% mentioned that the role of the teacher is that of a facilitator that
only guides, while the learners are more in control of their learning and the pace of the lesson,
thus allowing them to learn better from their own experience and through experience sharing.
Another participant said:“The teacher guides but the learner is more in control of the path their learning
takes”.
The least popular responses (5%) were that an experiential approach encourages a flippedclassroom model, collaboration in groups or teams, and that it helps students achieve both
personal and learning outcomes that are meaningful. A few participants also mentioned that
this approach represents equal opportunities for both advanced and less advanced learners, inthat it stimulates critical thinking, decision-making and problem-solving in all students.

156

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Question 3: How is experiential learning practiced? What types of activities initiate andreinforce experiential learning?

Figure 3: Applications of experiential learning
22% of the responses indicated that experiential learning is based on classroom-based learning
experiences such as presentations, case studies and problem-solving exercises. A high numberof participants also believed that it is based on group activities and teamwork (11%), as well
as reflection or activities that develop personal insights and understanding (17%).
Many answers (8%) suggested field-based learning experiences or internships and a flippedclassroom model as activities to encourage experiential learning. Another 8% described thatthe classroom should be used as a space to share information and discuss findings, and to create
experiences. 7% encouraged hands-on class activities which are real and engaging. One
participant said:“This type of learning can be practiced in a number of ways, both in the classroom
and in a real practical environment. With adult students this type of learning isvery useful as they would have a great deal of experience that can be shared”.
Question 4: Can you name some types of experiential learning activities?

Figure 4: Experiential learning activities

157

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Participants named outward bound activities such as fieldtrips (13%), role-plays or simulations(13%) and group activities (11%) as the most popular experiential learning activities. One
participant said:“Short case studies with a problem-solving element, so that learners are able toconceptualise, followed by a role-play like application in group work
environment, create field-based activities, site visits, reflecting on common
problems in the workplace and work in groups to solve the problems, then present
the findings”.
Many answers suggested internships or volunteering (10%), class discussions or debates (9%),
case study analyses (8%) and reflective writing and poster presentations (7%). A less popular
activity mentioned was drama (5%), as a more elaborate form of role-play.
Question 5: What is the teacher’s role in experiential learning?

Figure 5: Teacher’s role in experiential learning
19% of the responses pointed out that the teacher is a guide who supports students inexperiential learning activities. One participant gave an example:“The teacher is a facilitator and will offer guidance. It will be important to set thescene with special attention to cultural sensitivities. I used an exercise called
Chinese whispers, commonly used in schools and a fun exercise. The leader needs
to have enthusiasm and motivation to make this type of learning happen, and thewhispers did, it clearly shows how leaders should not communicate”.
Many responses (12%) showed that the teacher’s role in experiential learning is to design anddeliver experiences. Some other responses (9%) highlighted the importance of reflection inexperiential learning, and 8% suggested that engaging students in decision making and problem
solving helped facilitate an experiential learning environment.

158

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Lastly, 5% of the responses revealed that the teacher’s role in experiential learning is to provide
feedback and answer students’ enquiries. The participants discussed that providing feedback
and answering students' questions would enable them to experience critical thinking attributes.
Question 6: How is learning assessed in this approach?

Figure 6: Assessment of experiential learning
The highest percentage of the responses (23%) showed that experiential learning can beassessed through reflection. It was found that the reflective assessments made it easier forlearners to measure their progress over the learning period and evaluate their success. One
participant said:“The new learning and the growth in knowledge past the experiential learning
activities through the self-reflection process shall be an integral part of theassessment. Towards the end of the experiential learning activity experience, thestudent should be able to describe/document how he will use the new learning in
a new environment facing a similar situation”.
The second popular assessment method was through self-evaluation (15% of the responses
received). Participants pointed out that self-evaluation-based assessment is a powerful way ofmeasuring experiential learning at any stage:“Learners benefit from the personal and flexible assessment opportunities
received from self-evaluation”.
The responses showed that the assessment of experiential learning should be conducted
throughout the learning journey (11% of the response rate). Measuring experiential learning
through group work was chosen in 10% of the responses. Similarly, another 10% of the
responses received showed that project-based assessment could also measure experiential
learning. 7% of the responses revealed that experiential learning can be assessed within theexperiential learning process itself.

159

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 20195% of the answers showed that one-on-one evaluation, capability measurement systems andletting students choose their preferred assessment methods are important in assessing
experiential learning outcomes.
Question 7: What is the place of reflection in learning?

Figure 7: The importance of reflection in experiential learning
The highest percentage of the responses (21%) showed that reflection should be practiced
during experiences. Participants said that the students and teachers should think about what
they have done, reflect on what has been learnt or achieved and then modify their behaviour asa result. One participant said that reflection should be conscious and productive:“Reflection will naturally occur from experience. The question is whether it is left
to haphazard chance of the unconscious which may lead to unhelpful connections
or not be at all effective, versus getting the reflective activities to occur at a more
conscious level and be open to feedback to assist that process to build healthy
productive reflective practices at the conscious level that builds a better quality
learning cycle”.
Another 21% of the responses received revealed that when new knowledge and skills are
gained, reflection should be part of the process. Participants said that the reflection process
helps the learners to apply the learned principles to a new similar situation. The third answer
with 21% popularity was that reflection should be practiced throughout the learning life cycle,
joining the past and present learning and consequently preparing them for the future.
Some responses (11%) indicated that reflection is done through meditation as an active,
conversive, dialectical exercise that requires as much intellectual work as does every otheraspect of the learning process, from analysis to synthesis to evaluation.

160

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Question 8: What are the ways in which reflection may be practiced?

Figure 8: Examples of reflective practice in experiential learning
The most popular response (24%) was that reflection should be practiced through reflective
journals with a focus on concepts, feelings, thought processes and the possible application of
new learning. 18% of the responses said that reflection should be practiced via strong mental
engagement throughout the past, present and future.
13% of the answers suggested that reflection be practiced through social and peer learning.
Reflective activities such as personal reflections, reflective assessments, and learning fromexperiences were other popular responses (8%). A few responses (5%) revealed that reflection
can be practiced through questioning.
Quantitative Data Analysis and Findings
The next step of the induction programme was the classroom observation process, where theexperiential learning team had the opportunity to observe the new teaching staff in order tomeasure to what extent experiential learning activities were practised at our tertiary institution.
Figure 9 represents the statistical analysis of the results from all the class observations. It showsthat 27% of the class activities observed were teacher-centred and 73% of them were learnercentred. Giving feedback on learners’ ideas (22%), engaging in practical experiential learning
activity (11%), listening to student presentations/ideas (10%) and teacher responding to student
questions (10%) were the four main leaner-centred activities observed. Reflective practicerepresented 1% of the activities.
Observers generally remarked that classrooms were well-equipped and arranged to facilitate
interaction. All students were reported to have the necessary materials for instruction. Goals
for sessions were adequately communicated and learning was guided. Moreover, most of theclass observations showed that teachers used real world examples and case studies, together
with a wide variety of activities such as in-class team building exercises.

161

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Figure 9: Statistic results of class observations

Figure 10: Legend of teacher-centred and learner-centred activities on the observation toolDiscussion
This study intended to share the findings of a trial staff induction programme designed topromote experiential learning and reflective practice. Based on literature, which promotes
experiential learning and reflective practice, the role of the staff induction was very importantfor us to be able to implement these learning and teaching methods in the curriculum of ourinstitution. Throughout the induction programme, our staff showed a high interest in exploring
experiential learning and reflective practice.
The results of the online induction course showed that staff were aware of learning throughexperiences and reflective practice. Overall, the findings supported the experiential learningtheories provided by Boydell (1976), Chickering (1977), Saddington (1992), Cantor (1997),
Jarvis (1999), and Beard and Wilson (2002). The findings specifically showed that employing
experiential learning and reflective practice brings an authentic learner-centred environment

162

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

into the classroom (Beard & Wilson, 2015; Moon, 2004). Therefore, our results support thevalue of academic staff induction programmes in order to promote student-centred teachingpractice, allowing students to play an active role in their learning to successfully achieve higher
learning outcomes (Yuen & Hau, 2006).
The responses indicated that, in experiential learning, the teacher’s main role is to guide andsupport students. Moreover, they reported that the teacher also plays a central role in engaging
students in decision making, problem solving and providing feedback and answers to students'
enquiries, so that students would eventually develop their own values, attitudes, knowledge
and skills throughout their learning cycle (Byram & Dube, 2008). The participants noted that,in order to successfully guide and facilitate these activities, the teacher needs to develop therequired experience to deliver these types of classes. Overall, the results of this study reinforce
that staff induction programmes play a key role in supporting teachers to adopt a more learnercentred role, which promotes student engagement.
The participants reported that experiential learning means a hands-on approach in which
students learn by doing and reflecting. It can continue throughout the life cycle of the learningjourney, which is aligned with Kolb’s experiential learning cycle. The participants’ experience
also supported Chadha and Nicholls (2006) statement that experiential learning activities makestudents more responsible for their learning while enhancing their capabilities and transferable
skills. Their responses supported that the key characteristics of experiential learning include: a
student-driven nature, a practice-centred approach, active student engagement, as well as
problem solving and reflective tasks that are connected to the real world. Among experiential
learning activities suggested by the participants in the questionnaires, field trips, role-pays,
simulations, group activities, volunteering/internships, class discussions/debates, reflective
writing and presentations were dominant.
Based on their responses in the online questionnaire, the participants believed that experiential
learning is mainly assessed through reflection. They believed that as part of reflective
assessment, self-reflection should be considered an integral part of the assessment process
throughout the learning journey. The participants also thought that another effective way ofmeasuring learning through experience is group work, as it promotes inclusiveness, teamwork,
leadership and communication. The study showed that it is important to embed reflection as an
assessment method for evaluating the experiential learning process. It provides opportunities
for self-assessment and lifelong learning (Bond, Evans, & Ellis, 2011).
The results of the class observations confirm the participants’ understanding of the importanceof learner-centred classroom activities, such as giving feedback on learners’ ideas, engaging inpractical experiential learning activities, listening to students' presentations/ideas andresponding to learners' questions. The findings reinforced what staff already knew about
experiential learning and reflection and showed that they managed to use experiential learningin their classes. However, the class observation data analysis showed that only a small numberof reflection activities were used during classes and that many teachers were not familiar withreflection and its implementation with the students. Even though experiential learning activities
filled 11% of observed class time, reflective practice occupied a mere 1%. It became apparent
that, although theoretical awareness about experiential learning was not a problem for new
academic staff, the practical application of this was not always fully achieved. This wasprobably due to the fact that most teaching practitioners saw experiential learning simply as
isolated class activities, exercises from which students can learn. A deeper approach toexperiential learning, one that shows that an experiential learning activity is only complete

163

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

when it is accompanied by a short reflective task, was not fully understood. This researchsupported the idea that further professional development workshops might need to be
conducted at this institution, to discuss how experiences can be reflected upon, in order toenable every teacher to use reflection appropriately.
Overall, in contemporary educational practice, academic staff induction programmes can help
create better staff relationships through pre-observation meetings, class observations and postobservation meetings. Making teachers aware of the collected data on their performance helps
to inform staff future goals and professional development needs. Teacher performance data
could also help educational providers report on the overall learning and teaching performance.
It provides clear information on what teachers actually do in the classroom and how that aligns
with organisational values, goals, student needs and funding requirements. This should
hopefully lead to better quality teaching and produce better quality graduates.
Conclusion
The aim of this particular induction programme was to encourage more teaching staff to use
an experiential learning classroom environment, as part of the institution’s general teachingpolicy. As highlighted in the literature review, experiential learning and reflective practice areboth relevant for current education theories and research, so we conducted this study in orderto help the institution to change its learning and teaching methods from the old teacher-centred
way to a more experiential approach. The research revealed that staff induction is a successfulprocess that generates mutually beneficial critical conversations between educational
professionals.
In theory, academic staff understood what experiential learning is and its overall importance intertiary education, compared with other more conventional learning approaches. During our
discussions, the staff proved to be very familiar with different types of learner-centred
experiential learning activities and seemed keen to trial them in their own classes. Most
participants in the questionnaire suggested that reflection plays a major role in experiential
learning and that this is one of the main ways in which we can assess experiential learning.
They also came up with a variety of useful reflective tasks that can be practiced in class.
In practice, following the class observations, we were able to confirm our initial findings thatteaching staff were very creative in using experiential learning activities in their classes andmaking sure that students are having a fun time, trialling a wide variety of practical exercises.
However, even though most of the teaching staff knew the theory behind experiential learning(or have found out and consolidated this theory during the short Moodle course andquestionnaire), not everyone understood how to achieve real learning through an experiential
learning type of lesson.
Overall, this process has proven to be very beneficial for both new staff and the experiential
learning team, as this is a great opportunity for everyone to share interesting and original ideas
about experiential learning, learn from each other and reflect more on their practice. Based onthese findings, we concluded that a better-organised staff induction programme at our
institution is key to further increase the quality of learning and teaching.

164

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Limitations of this Study
We acknowledge the limitations of this study. Firstly, the results of the study might appear to
not reflect the general situation at our institution, as the number of participants was limited to
16 (due to only having 16 new staff who went through the induction process during the study
period). However, the sample was found to be representative for the size of our institution.
Secondly, the class observations conducted were all scheduled, giving staff the opportunity toplan their lessons in advance. Therefore, the classroom behaviour observed may not haverepresented their usual teaching habits. Thirdly, the results of the observations could have beenfurther affected by the stress of being observed, as well as the time limitation of 30 minutes per
observation.
Recommendations
The paper aims to highlight the need for creating a cultural shift with improved learning styles,
implementing experiential learning and reflective practice among staff and students in thetertiary education sector. The new interesting ideas generated on experiential learning andreflective practice during the research are certainly good resources to apply and practice in theclassroom, although this is just the beginning of the process. A more sustained and constructive
approach from all the parties involved should be beneficial for better results in the future(institution, academic staff and students).
The willingness of the institution to promote and support this programme is necessary.
Implementing experiential learning and reflection practice into the usual staff induction
process, to include education and training on the importance of these teaching strategies, would
definitely be beneficial. Introducing these strategies into the existing curriculum would require
extra work on curriculum design, as well as a specific implementation and a guided assessment
process.
Training sessions should be provided to both staff and students on adjusting the traditional
teaching and learning methods by introducing more experiences and reflection. The results
show that staff might benefit from more training in terms of reflective activities they canemploy during an experiential learning lesson, to make sure that this is indeed a meaningfullearning experience. Therefore, support services should be in place to guide, support andevaluate the progress for both staff and students. These services should be able to guide andhelp students to continue learning through experiences and to continuously reflect in theirself-directed learning journey.

165

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
Ahmed, E., Nordin, Z. S., Shah, S. R., & Channa, M. A. (2018). Peer observation: A
professional learning tool for English language teachers in an EFL institute. World
Journal of Education, 8(2), 73−87. https://doi.org/10.5430/wje.v8n2p73
Bear, G. G., & Minke, K. M. (eds.). (2006). Children’s needs III: Development, prevention,and intervention. Washington, DC: National Association of School Psychologists.
Beard, C., & Wilson, J. P. (2002). Experiential learning. London: Kogan Page.
Beard, C., & Wilson, J. P. (2015). Experiential learning: A handbook for education, training
and coaching. (3rd ed.). London: Kogan Page.
Bond, J. B., Evans, L., & Ellis, A. K. (2011). Reflective assessment. Principal
Leadership, 11(6), 32−34.
Boud, D. (1994). Conceptualising learning from experience: Developing a model for
facilitation. Proceedings of the 35th Adult Education Research Conference, 49–54.
Boud, D. (2010). Sustainable assessment: Rethinking assessment for the learning society.
Studies in Continuing Education, 22(2), 151−167. https://doi.org/10.1080/713695728
Boydell, T. (1976) Experiential learning, Manchester Monograph 5, University ofManchester, Department of Adult Education.
Brookfield, S. D. (1995). Becoming a critically reflective teacher. San Francisco: Jossey
Bass.
Brookfield, S. D. (2017). Becoming a critically reflective teacher (2nd ed.). CA: Jossey-Bass
Inc.
Bryman, A. (2016). Social research methods (5th ed.). Oxford NY: Oxford University Press.
Byram, R., & Dube, H. (2008). Planning for Success. NY: Continuum International
Publishing Group.
Cantor, J. A. (1997). Experiential learning in higher education: Linking classroom andcommunity. The George Washington University, Graduate School of Education and
Human Development, Washington DC
Cavilla, D. (2017). The effects of student reflection on academic performance andmotivation. SAGE Open, 7(3). https://doi.org/10.1177/2158244017733790
Chadha, D., & Nicholls, G. (2006). Teaching transferable skills to undergraduate engineering
students: Recognising the value of embedded and bolt-on approaches. International
Journal of Engineering Education, 22(1), 116−122.
Chickering, A. W. (1977) Experience and learning: An introduction to experiential learning.
Change Magazine Press, New Rochelle, NY
Dewey, J. (1938). Experience and education. NY: Collier.
Felicia, P. (2011). Handbook of research on improving learning and motivation througheducational games: Multidisciplinary approaches. Hershey, Pennsylvania: IGI
Global.
Gardner, H. (1999). Intelligence reframed: Multiple intelligences for the 21st century. New
York: Basic Books.

166

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Jarvis, P. (1999) International dictionary of adult and continuing education. Kogan Page,
London.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning anddevelopment. New Jersey: Prentice Hall.
Kolb, D. A. (2015). Experiential learning: Experience as the source of learning anddevelopment. New Jersey: Pearson Education (US).
Lewis, L. H., & Williams, C. J. (1994). Experiential learning: Past and present. New
Directions for Adult and Continuing Education, 62, 5−6.
https://doi.org/10.1002/ace.36719946203
Miller, R. J., & Maellaro, R. (2016). Getting to the root of the problem in experiential
learning: Using problem solving and collective reflection to improve learning
outcomes. Journal of Management Education, 40(2), 170−193.
https://doi.org/10.1177/1052562915623822
Moon, J. A. (2004). A handbook or reflective and experiential learning: Theory and practice.
Oxon: Routledge Falmer.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implicationsof combining qualitative and quantitative methods. Journal of Mixed Methods
Research, 1(1), 48–76. https://doi.org/10.1177/2345678906292462
Murphy Tighe, S., & Bradshaw, C. (2013). Peer-supported review of teaching: Making thegrade in midwifery and nursing education. Nurse Education Today, 33(11),1347−1351. https://doi.org/10.1016/j.nedt.2012.07.016
Pedler, M. (2011). Action learning in practice. Surrey, England: Gower Publishing Limited.
Rashidi, N., & Moghadam, M. (2014). The effect of teachers’ beliefs and sense of selfefficacy on Iranian EFL learners' satisfaction and academic achievement. e Electronic
Journal for English as a Second Language, 18(2).
Saddington (1992). Learner experience: A rich resource for learning. In J. Mulligan, & C.
Griffin (eds.), Empowerment through experiential learning: Explorations of goodpractice, 37−39. Kogan Page, London.
Schon, D. A. (2017). The reflective practitioner. NY: Basic Books Inc.
Schonell, S., Gilchrist, J., Kennelly, R., McCormack, C., Northcote, M., Ruge, G., & Treloar,
G. (2016). TATAL talking about teaching and learning: Teaching philosophy
workbook. Hammondville, Australia: Higher Education Research and Development
Society of Australasia.
Stephenson, J. (1998). The concept of capability and its importance in higher education.
Capability and Quality in Higher Education, 1−13. London: Kogan Page.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes.
London: Harvard University Press.
Yuen, K. M., & Hau, K. T. (2006). Constructivist teaching and teacher‐centred teaching: A
comparison of students’ learning in a university course. Innovations in Education andTeaching International, 43(3), 279−290. https://doi.org/10.1080/14703290600750861
Corresponding author: Vera Maria Nistor
Contact email: vera.nistor@op.ac.nz

167

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019168

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Independence and Interdependence of a Professional Development School Partnership
Robert F. Sumowski
Georgia College & State University, USA
Joseph M. Peters
Georgia College & State University, USA

169

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Abstract
Professional Development School Partnerships have the potential to provide uniqueexperiences for teacher candidates and also provide important professional development for
the K-12 school personnel involved in the relationship. According to the National Associationof Professional Development Schools (2008), a key essential to a successful partnership
involves the entities’ ability to achieve objectives together that neither partner might be able tofully accomplish individually. Our Professional Development School is a model for how thepartners’ joint mission operates beyond each institution’s ability to provide the experiences andservices in isolation. Although both the school district and the university are independent
entities, their interdependence in the Professional Development School allows us to accomplish
the overarching mission to improve education at all levels in our community from six weeks
old to the doctoral level. The benefits are initiatives such as a joint Montessori Academy forchildren, participation on advisory councils, and a grade seven through twelve Early College
program. Classrooms are made available for university faculty to do research, preservice
teacher candidates can observe and practice, and first-generation at-risk students have thesupport they need for college placement. Professional development offerings by university
faculty and District staff in areas such as Positive Behavior Interventions and Supports allow
District teachers a unique professional development experience and the ability to continue andachieve a university certificate for their participation.
Keywords: professional development school, positive behavior interventions and supports,
independence, interdependence

170

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Over 30 years ago, the Holmes Group introduced the concept of Professional Development
School (PDS) relationships as “working partnerships among university faculty, practicing
teachers, and administrators that are designed around the systematic improvement of practice”(1986, pp. 66-67). They likened them to teaching hospitals, where research and practiceexchanges occur that benefit both entities. Since that time, PDS relationships have grown froma “concept to a vibrant and important reality;” “a cornerstone of serious attempts tosimultaneously improve teacher education and public schools” (Teitel, 2004, p. 1). Today the
National Association for Professional Development Schools (n.d.; 2019) supports an annual
conference and the School-University Partnerships journal. In this publication, the authors
highlight the importance of the interdependence of the relationship while still maintaining
University and District independence. Various examples, such as instituting a Positive
Behavior Interventions and Supports (PBIS) program, a Montessori Academy, an Early
College Program for middle and high school students, and multiple joint grants, illustrateexamples of the independent and interdependent relationship.
The purpose of the present study was to examine the various aspects of the PDS partnership
between a University and its neighboring PreK-12 school district in light of its independentand interdependent relationship utilizing a mixed methods design. The study is relevant to thosestudying or planning to implement similar partnerships in that it illustrates successes andchallenges within the PDS model. Following a review of relevant literature, the researcherspresent an outline of the methodology that guided the study, results, and a discussion of the,the study’s limitations, and suggestions for further research. Four categories, including (1)adhering to essentials guiding the partnership, (2) reading and mathematics literacy support,(3) developing and supporting joint programs, and (4) positive behavior interventions andsupports are explored as they relate to the independent and interdependent relationship andstudent and district success.
Establishing the Relationship
The PDS partnership was developed in 2015 between a state university and a county-wide
charter school district in central Georgia. It was shortly after the District’s superintendent andthe College of Education’s dean were hired. Both had previous experiences with PDS
partnerships in other settings. After initial exploratory discussions and guidance from external
PDS consultants, the University and District developed a memorandum of understanding(MOU) and assigned a liaison from each entity to coordinate monthly PDS meetings to guideand coordinate the partnership’s many activities. The liaison role was seen as essential to thepartnership (Parker, Parsons, Groth, & Brown, 2016).
Because literacy was an initial priority, a literacy faculty member represented the College of
Education (COE), spending a day each week in the District. As the partnership progressed, asecond faculty member with PBIS experience was also assigned to the District. In addition to
the monthly PDS meetings, the COE Dean, Georgia College & State University (GCSU)
President and Provost, and the District Superintendent met on an annual basis to review thepartnership and look at potential opportunities that fit within the guiding essentials of a PDS
relationship.
Literature Review
Professional Development Schools originated with the goal of creating school-university
partnerships for educational change through enhanced professional development (DarlingHammond, 1994, 1998; Goodlad, 1990; Holmes Group, 1986, 1990; Levine, 1992, 1998).

171

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Professional development is enhanced when there is a supporting environment (Chang, Chen,& Chou, 2017; Tracz, Beare, & Torgerson, 2018). These partnerships are designed to promote
professional development of both the novice and veteran teacher (Carpenter & Sherretz, 2012;
Darling-Hammond, 1994; Ismat, 1999; Nzarirwehi & Atuhumuze, 2019). The latter is often
overlooked however, and there can often be a lack of parity (Al Seyabil, 2017; Breault, 2014).
Oftentimes the impetus for the relationship is that colleges need places to place preservice
teachers (Walsh & Backe, 2013). One goal of our partnership and supporting research was tochallenge the notion that these relationships are limited to just professional development and aplace to assign preservice teachers.
As the American Association of Colleges for Teacher Education (AACTE) points out“[a]nother important teaching function of colleges of education is providing continuing
professional education for teachers, school administrators, and other education professionals”(2018a, p. 37). Darling-Hammond (1998) emphasizes the duality of the relationship bystressing that a professional teacher continues to learn from teaching.
These new programs typically engage prospective teachers in studying researchand conducting their own inquiries through cases, action research, and structured
reflections about practice. They envision the professional teacher as one who
learns from teaching rather than as one who has finished learning how to teach,and the job of teacher education as developing the capacity to inquire
systematically and sensitively into the nature of learning and the effects ofteaching (Darling-Hammond, 1998, p. 9).
PDS provides a role for teacher educators to assist both novices and experts with the importantend goal of promoting student achievement, an often-overlooked area of the partnership(Abdal-Haqq, 1999; Darling-Hammond, 1998; Darling-Hammond, Ramos-Beban,
Altamirano, & Hyler, 1998; Murrell, 1998, Renyi, 1996). According to the National
Association for Professional Development Schools (NAPDS) vision, there needs to be “equal
representation of, and access for, all educators invested in improving education” (2008a). One
primary goal of our partnership and supporting research was to challenge the notion that these
relationships are limited and simply facilitate things that occur without a PDS relationship.
Professional development work in the schools also promotes Boyer’s concept of thescholarship of engagement as faculty share their expertise in the schools (Andrews & Leonard,
2018; Boyer, 1996). This can be challenging at times since successful partnerships involve
collaborative relationships that support school-wide approaches with every teacher involved(Colwell, MacIsaac, Tichenor, Heins, & Piechura, 2014; Parsons & Renyi, 1999). Additionally,
there can also be challenges to the role the teacher educator plays in the school, how thehierarchy is established, and when there are organizational changes (Ronen, 2018; Schack,2011). One area for our partnership was classroom management through positive behavioral
approaches.
Early College Program
In Georgia, Early College is a partnership between the Georgia Department of Education andthe University System of Georgia. Public schools partner with public colleges and universities
to offer high school courses and dual enrollment opportunities. According to the Board ofRegents of the University System of Georgia (2017, para. 2), “[e]arly College is an intervention
strategy for students who may not be well served by traditional high schools. The initiative also

172

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019seeks to change the face of higher education by targeting those students traditionally
underrepresented on college campuses.”Montessori Academy
Montessori education was developed by Maria Montessori, an Italian physician. It dates backmore than 100 years. It is characterized by “children working independently and in groups,
often with specially designed learning materials; deeply engaged in their work; andrespectful of themselves and their surroundings” (American Montessori Society, n. d., para.
2). Another distinguishing feature is that “children are provided freedom to explore subjects inblended classrooms where older students solidify their own knowledge by helping the younger
students learn” (Montessori Academy at the Early Learning Center, n. d., para. 1).
Positive Behavioral Interventions and Supports
School-wide Positive Behavioral Interventions and Supports (SW-PBIS) is “a systems
approach to establishing the social culture and behavioral supports needed for all children in aschool to achieve both social and academic success” (Horner, Sugai, & Lewis, 2015, p. 1). SWPBIS initially involves a leadership team where behavior specialists, administrators, regularteachers, and special needs teachers attend training sessions on PBIS, establish schoolwide
goals, and begin an implementation plan (Netzel & Eber, 2003). This is followed by schoolwide implementation, where the team provides ongoing professional development for thebuilding staff, analyzes the behavioral policies already in place, adjusts as needed, and begins
documenting outcomes with a research-based approach (Cressey, Whitcomb, McGilvrayRivet, Morrison, & Shander-Reynolds, 2018; Hirsch et al., 2019; Horner et al., 2014; Horner,
Sugai, & Lewis, 2015; Netzel & Eber, 2003; Sugai & Horner, 2009). The core elements of
PBIS are shown in Table 1.
Table 1: Core elements at each of the three tiers in the PBIS prevention model
Prevention Tier
PrimaryCore Elements
Behavioral Expectations Defined
Behavioral Expectations Taught
Reward system for appropriate behavior
Clearly defined consequences for problem behavior
Differentiated instruction for behavior
Continuous collection and use of data for decision-making
Universal screening for behavior support
Secondary
Progress monitoring for at risk students
System for increasing structure and predictability
System for increasing contingent adult feedback
System for linking academic and behavioral performance
System for increasing home/school communication
Collection and use of data for decision-making
Basic-level function-based support
Tertiary
Functional Behavioral Assessment (full, complex)
Team-based comprehensive assessment
Linking of academic and behavior supports
Individualized intervention based on assessment information focusing on (a)prevention of problem contexts, (b) instruction on functionally equivalent skills, andinstruction on desired performance skills, (c) strategies for placing problem behavior
on extinction, (d) strategies for enhancing contingence reward of desired behavior,and (e) use of negative or safety consequences if needed.
Collection and use of data for decision-making
Reference: Horner, Sugai, & Lewis, 2015, p. 2

173

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

The National Association of Professional Development Schools Essentials
The National Association of Professional Development Schools (NAPDS) provided a policy
statement titled “What It Means to Be a Professional Development School,” at its annual
meeting April 12, 2008 (2008b). This statement, based on the work from a summit the prioryear, delineated what it meant to have a PDS relationship by providing guidelines that wereshown as the “nine essentials” (2008a). The purpose of this document was to specificallydelineate what a professional development school relationship was and eliminate the “catchall” models that were not true PDS relationships. In the document, they provided the nine
required essentials of a PDS as:
1. A comprehensive mission that is broader in its outreach and scope than the mission of
any partner and that furthers the education profession and its responsibility to advanceequity within schools and, by potential extension, the broader community;
2. A school–university culture committed to the preparation of future educators thatembraces their active engagement in the school community;
3. Ongoing and reciprocal professional development for all participants guided by need;
4. A shared commitment to innovative and reflective practice by all participants;
5. Engagement in and public sharing of the results of deliberate investigations ofpractice by respective participants;
6. An articulation agreement developed by the respective participants delineating theroles and responsibilities of all involved;
7. A structure that allows all participants a forum for ongoing governance, reflection,and collaboration;
8. Work by college/university faculty and P–12 faculty in formal roles acrossinstitutional settings; and9. Dedicated and shared resources and formal rewards and recognition structures.
(2008b, pp. 2-3).
These essentials are all important to the proper functioning of a PDS relationship.
Evaluating the Success of a Professional Development School
When referring to the evaluation of the long-term professional development school relationshipat the University of North Carolina at Wilmington and its partners, the authors note the lack of
empirical studies on PDS models (Lewis & Walser, 2016). Bentley and Hendricks (2018)
qualitatively discuss their collaborative experiences in promoting literacy, but do not provide
the overall partnership specifics. Polly, Reinke, and Putnam (2019) explore their partnership inlight of Goodlad’s postulates (Goodlad, 1994) and the AACTE clinical practice report(American Association of Colleges for Teacher Education [AACTE], 2018b). They address thenine essentials, but the article is essentially a synthesis of the NAPDS, AACTE, and Goodlad’s
postulates.
Many studies discuss the PDS partnership from a university perspective, but often only focus
on the field experiences for preservice teachers. Darling-Hammond (2010) cites positive
experiences for teachers, but as M. Caparo, R. Caparo, and Helfeldt report, studies are“equivocal as the learning that occurs during field experiences” (2010, p. 132). This is due tothe contextualization of programs involving specific partners. Darling-Hammond & Bransford(2005) have documented gains in student performance resulting from PDS initiatives (seepages 415–416). This study looks at the PDS relationship, support for the nine essentials, andthe outcomes of a strong PDS.

174

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Methodology and Methods
Participants
The partnership is with a county charter school system in rural Georgia. It is a Title I school
district with four elementary schools, one middle school, and one high school. The student
population is representative of the county's population. Over 80% of the students qualify forfree or reduced-price meals and there is an approximate enrollment of 5,500 students. The
racial makeup is 65% black, 28% white, 1% Asian, 2% Hispanic, and 4% multi-racial.
Research Design
Because of the various components of the PDS relationship, a mixed methods approach wasused to study the various outcomes of the partnership. A variation of the explanatory design
was used because the quantitative and qualitative data was collected in separate phases andlater used to support the research questions (Creswell & Plano Clark, 2006). Johnson,
Onwuegbuzie, and Turner (2007) describe mixed methods research as follows:
Mixed methods research is the type of research in which a researcher or team ofresearchers combines elements of qualitative and quantitative research
approaches (e. g., use of qualitative and quantitative viewpoints, data collection,
analysis, inference techniques) for the broad purposes of breadth and depth ofunderstanding and corroboration (p. 123).
Table 2 shows the four main areas we used to examine the PDS partnership and why they wereincorporated into the design.
Table 2: Areas of support
Four areas used to study the partnership and why they were chosen
1. Adhering to
2. Reading and3. Developing and
4. Positive Behavior
essentials guiding
mathematics literacy
supporting joint
Interventions andthe partnershipsupportprograms
Supports
Because this was an
One of the first partner
Two major ongoing
PBIS was the second
emerging partnership,
activities was to provide programs are the Early major area of partner
both groups wanted toliteracy support. This isCollege where grades
support after thehave a highly successful a key evaluative area for 7-12 students attend
literacy support. This
professional
schools in Georgia (Theregular and college
area provided the ability
development school
College and Careerclasses on thefor the district torelationship, and because Readiness Performance
university campus and become a “PBIS
both entities had PDS
Index that ranks schools a Montessori
district” with therelationships with otherand includes literacy
Academy for birthGeorgia Department ofinstitutions with various data as measured by thekindergarten children
Education, having alevels of success, it was
Georgia Milestonesof parents from thespecial education
important to adhere tostatewide exam). It is
district and university
professor familiar withthe to nine essentials as
also an area that washave childcare in the
PBIS on site to provide
outlined by the National initially determined as
district’s early
professional
Association forneeding support.
learning center. We
development and also
Professional
Positive results in thiswere especiallyinvolve the special
Development Schools.
area would show that the interested in looking at education preservice
Since this would be
PDS relationship was
how the partnershipteachers was a win fordifficult to explore in abenefiting the schools
would help facilitate
both partners and aquantitative way, aand participatingproblem solving when reason to look at if there
qualitative approach was preservice teachers.
there were problems at was success in that area
taken.
either partnership site. of the PDS partnership.

175

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 3 illustrates the components of the design with the research methodology and questions.
Table 3: Research questions and methodology
Research Question
Did the University and theDistrict adhere to nineessentials as outlined by the
National Association forProfessional Development
Schools?

Research Method
Qualitative evaluation research(Denzin & Lincoln, 2011;
Newman, 2011) conducted byinternal evaluators looking at
the nine required essentials by
the National Association forProfessional Development
Schools (National Association
for Professional Development
Schools [NAPDS], 2008b) incomparison to the outcomes ofthe partnership. Interviews
were the primary source ofdata when comparing thestandards to the practices.

Will there be overall gains inelementary school student
achievement (e.g. literacy,
math, and science) as measured
by the school’s state grade on
the College and Career Ready
Performance Index (CCRPI)
that includes the Georgia
Milestones end-of-grade
assessment in reading andmathematics?
Will joint projects be mutually
beneficial to both partnerorganizations?

Comparing 2014 baseline
College and Career Readiness
Performance Index (CCRPI)
scores to 2017 scores with acorrelated two-tail t-test(Lowry, 2019).

Will the Positive Behavior
Interventions and Supports
program lead to a significant
reduction in discipline
incidents reported to the state?

Comparing 2014 (baseline
Office Disciplinary Referrals(ODR) scores to 2016 scoreswith a correlated two-tailed ttest (Lowry, 2019).

Qualitative evaluation research(Denzin & Lincoln, 2011;
Newman, 2011) conducted byinternal evaluators looking at
the outcomes of the partnership
through interview data anddiscussions.

176

Design References“The systematic
identification and assessmentof effects generated bytreatments, programmes,
policies, practices, andproducts.” (Jupp, 2009, p.
104).
“At the most elemental level,
evaluation research is aimed atdetermining whether a program
was actually carried out. At amore complex level, researchis concerned with theeffectiveness and/or the
economic attributes of aprogram. Ideally suchinformation should contributeto decisions about whether toexpand, curtail, or modify aprogram.” (Gordon & Morse,
1975, p. 339)
“The t-test for correlated
samples is especially useful inresearch involving human or
animal subjects precisely
because it is so very effective
in removing the extraneous
effects of pre-existing
individual differences”(Lowry, 2019).
The conclusions can lead to “anew design and development
model” or “an enhanced
version of an existing designand development model”(Richey & Klein, 2007, p.
136). Supporting data includes“user perceptions and reports.”

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Ethical ApprovalInstitutional Review Board approval was provided to study the Professional Development
School relationship with the district. In this study, only top-level, public data were used so that
individual teachers or students could not be identified.
Results and Discussion
The findings can be grouped into four categories that reflect the research questions. These
include (1) adhering to essentials guiding the partnership, (2) reading and mathematics literacy
support, (3) developing and supporting joint programs, and (4) positive behavior interventions
and supports. The narrative also provides information on what occurred as part of thepartnership.
Category 1: Adhering to Essentials Guiding the Partnership
Our PDS partnership is guided by the nine required essentials developed by the Executive
Council and Board of Directors of the National Association for Professional Development
Schools (NAPDS, 2008b). All members of the partnership agreed to this claim.
Essential 1. The first essential is “[a] comprehensive mission that is broader in its outreach andscope than the mission of any partner and that furthers the education profession and its
responsibility to advance equity within schools and, by potential extension, the broader
community” (NAPDS, n.d., para 4). Both parties agreed that it was important to the region to
improve the education of PreK-12 students. Few local students were competitive in gaining
admission to GCSU, given that it is a selective institution in the state system (GCSU, 2016).
As the president of the university pointed out in a meeting, “[w]e need to work collaboratively
if we are going to change the educational situation in our community” (S. Dorman, personalcommunication, August 8, 2018). At that time a goal was set to seek funding to establish acenter for rural education to assist middle Georgia rural communities in advancing education,
especially with economically-challenged communities.
The formal PDS memorandum of understanding between the District and University reflects
the comprehensive mission essential and starts out with the following:
Professional Development Schools (PDSs) represent a partnership among schools,
universities, and community stakeholders. The Baldwin County School District andGeorgia College are committed to becoming a driving force in the community bytransforming P-20 education and developing students, educators, and leaders who thinkcritically and design innovative and creative opportunities within a democratic andglobal society. The mission of the professional development school partnership
between BCSD and GCSU is to improve P-20 education for our students, educators,
and leaders through collaborative engagement in teaching and learning (Georgia
College & State University & The Baldwin County School District, 2015, p. 1).
Essential 2. The second essential is “a school – university culture committed to the preparationof future educators that embraces their active engagement in the school community” (NAPDS,
n.d., para 4). The College of Education has a long history of having a strong teacher preparation
program. Preservice teachers spend 2-3 days a week in the field as juniors and complete a yearlong internship as seniors. The District is a primary place for placing students for active
engagement in the classroom (C. Garrett, personal communication, July 30, 2019). Currently,
the middle grades program preservice teachers are all placed in the district’s middle school.

177

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Likewise, the special education program is holding class in the district and those students areworking in the schools. The early childhood/elementary and secondary programs are also
placed in the district whenever possible.
Essential 3. The third essential is an “ongoing and reciprocal professional development for allparticipants guided by need” (NAPDS, n.d., para 4). Due to the high poverty level of thecommunity, all schools in the District are Title I. In 2015, three of the four elementary schools
were considered “Focus Schools” by the State of Georgia due to not reaching minimum
achievement levels (Baldwin County School System, 2015). This put literacy professional
development for the District teachers as a high priority. It also offered additional professional
development opportunities for the preservice teachers assigned to the schools. We continue tosupport this effort and the scores on the state of Georgia Milestones exam reflect that the effort
is working.
Essential 4. The professional development opportunities also met the fourth essential of a“shared commitment to innovative and reflective practice by all participants” (NAPDS, n.d.,para 4). This was seen as classroom teachers tried new methods to engage students and provide
positive behavioral supports. Likewise, preservice teachers gained valuable insight into ways
to manage the complexities of the classroom. Working together, the partners were able tosecure state funding for the competitive Literacy for Learning, Living, and Leading (L4GA)
grants to cover things like sub pay and formal professional development for teachers.
Essential 5. The fifth essential is for “engagement in and public sharing of the results ofdeliberate investigations of practice by respective participants” (NAPDS, n.d., para 4). Sharing
the partnership outcomes continues to be an ongoing process with presentations at manynational and international conferences such as the International Academic Forum, the HawaiiInternational Conference on Education, National Association of Professional Development
Schools, and the International Conference of the Association for Positive Behavior Support.
One important outcome of the PBIS initiative was the assistance in the development of the
Georgia Positive Behavior Interventions and Supports (PBIS) strategic plan 2014-2020(Georgia Department of Education, 2014) and the associated rules that became the first everPBIS endorsement program for teachers (Georgia Professional Standards Commission, 2019).
Essential 6. The sixth essential is an “articulation agreement developed by the respective
participants delineating the roles and responsibilities of all involved” (NAPDS, n.d., para 4).
Early in the partnership, stakeholders met on several occasions to develop a comprehensive
articulation agreement which is reviewed each year and updated as needed. The roles were
outlined as seen below.
•

Georgia College Responsibilities:
o Goal 1: To improve educational opportunities for children enrolled in theschool:§ Provide human resources (teaching candidates) to lower teacher-pupil
ratios;§ Provide focused, differentiated instruction in tutoring and small groups
based on teaching candidate assignments;§ Assist in improving achievement levels;§ Prepare interns with methods reflecting best practices.
o Goal 2: To provide both informal and formal professional development for178

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Baldwin County teachers and Georgia College teaching candidates:§ Provide support in school-wide professional learning programs andactivities as requested;§ Facilitate in-classroom demonstrations and assistance as requested;§ Interact with school faculty in ways that promote professional growth.
o Goal 3: To administer teacher education standards and evaluation associatedwith certification:§ Communicate supervision and accreditation standards;§ Provide efficient procedures for evaluation requirements;§ Provide direction of intern mentoring and supervision.
•

Baldwin Schools Responsibilities:
o Goal 1: To provide exemplary teacher education opportunities for teaching
candidates:§ Model best practices in all curricular areas;§ Model differentiated instruction;§ Model progressive practices (discipline, inclusion, etc.);§ Mentor interns by supervising and providing feedback to planning andteaching;§ Provide supervised opportunities to develop independence in teaching;§ Adhere to standards and requirements associated with certification;§ Seek opportunities to collaborate with Georgia College faculty for
professional learning.
o Goal 2: To facilitate inquiry and research at the school site in accordance with
BCSD policy and Georgia College Institutional Review Board Policies:§ Assist in obtaining permissions for inquiry and research, in accordancewith federal statues and Baldwin County Schools policies;§ Facilitate requirements for Institutional Review Board approval;§ Willingness to engage in inquiry and support teaching candidates’
requirements;§ Provide access to institutional data and assessment/test data forresearch purposes, as legally permitted(Georgia College & State University & The Baldwin County School
District, 2015, pp.1-2).

Essential 7. Built into the articulation agreement is the seventh essential for “a structure thatallows all participants a forum for ongoing governance, reflection, and collaboration”(NAPDS, n.d., para 4). The University faculty and administrators, classroom teachers, andschool administrators continue to suggest new ways to enhance the partnership and meet on amonthly basis. With the addition of a high school teacher career academy, a new pathway to
becoming a teacher is now being explored where the University and District will partner toprovide a seamless entry into the teaching profession.
Essential 8. The eighth essential is “work by college/university faculty and P–12 faculty informal roles across institutional settings” (NAPDS, n.d., para 4). As the partnership developed,
the superintendent was awarded faculty status and invited to teach courses for the university.
Likewise, District and University faculty, teachers and administrators are involved in sharedinitiatives and committees.

179

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Essential 9. The ninth essential is “dedicated and shared resources and formal rewards andrecognition structures” (NAPDS, n.d., para 4). Teams worked together to identify resources touse in professional development and collaborative research. As it relates to recognition, theDistrict, Superintendent, and teachers have been recognized during award ceremonies at the
University and University personnel have been recognized by the District. Based on the nineessentials, there were many collaborative efforts that benefited both entities, and theparticipants all agreed that the nine essentials were followed.
Category 2: Reading and Mathematics Literacy Support
One of the first collaborative activities that we worked on was to apply for literacy grants to
support reading improvement in the District. This was determined by the superintendent andelected board of education to be an essential goal for the incoming administration. Because theDistrict, which serves a total of seven schools, did not have grants management staff, theUniversity’s grants center assisted in all aspects of the preparation and application process to
prepare the grant. In 2016, we were awarded the “K-2 Mathematicians and Writers” literacy
grant and began implementation of the literacy program within each of the P-12 schools in theDistrict. Supplemental assistance was provided through a University faculty member withexpertise in literacy who was reassigned one course per semester to work in the District, which
was paid from the grant.
Over the course of the next two years, the PDS partnership continued the “Read Baldwin
County” initiative. The designated faculty member continued to work within the District in its
implementation and eventually was assigned by the University to work full-time within theDistrict to support literacy and act as the PDS liaison between the partners. An additional
literacy faculty member supported the program in the summers.
This arrangement allowed the University to have teacher candidates’ field placements occur inclassrooms where literacy professional development and implementation was taking place at
the ground level. We could then test the theoretical concepts out with real students in a P-12
field-based settings, which resulted in a three-fold benefit to teacher candidates, the District,and its students. An added benefit to this arrangement was that it allowed the University tooffer a Reading Endorsement through state certification within the District.
As new opportunities arose, the partnership expanded its collaborative efforts. The next
opportunity was to work on Middle Grades science, technology, engineering, and mathematics(STEM) in a partnership that also included the arts called “STEAM Ahead”, which began in2017 and has continued through the submission of multiple subsequent grants. This effort
allowed the University to provide a faculty member a course reassignment to oversee thisSTEAM project which provided professional development to the District’s middle grades
teachers. We also brought our middle grades teaching candidates to the school for on-siteclasses and observations.
One thing PDS leaders learned through this process is that many granting agencies look for astrong partnership between PDS entities when making funding decisions. The strength of thecontinuing partnership between our two entities has played a role in the partnership’s ability tocontinue to obtain grant funding. Overall, there was a significant improvement in the District’s
College and Career Readiness Performance Index (CCRPI) scores when comparing schools
from the 2014 baseline to 2017 (see Tables 4 & 5). The CCRPI score reflects content masteryand include achievement scores in English language arts, mathematics, science, and social
studies based on student performance on the statewide Milestones assessments (Georgia

180

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Department of Education, 2019). Although the results are not significant, there is a positivegain. Note that the 2018 data cannot be used in any comparisons. The state of Georgia has
redone the CCRPI algorithm which prevents direct comparisons from previous years (see:https://www.gadoe.org/CCRPI/Pages/default.aspx).
Table 4: College and Career Readiness Performance Index (CCRPI) data
2014 CCRPI
71
71
55
52
59
66

BH Elementary
CS Elementary
ER Elementary
MW Elementary
OH Middle School
BH High School2017 CCCRPI
69
73
61
56
60
76

Note: Numbers are rounded off; Data source: http://ccrpi.gadoe.org

Table 5: County-wide College and Career Readiness Performance Index (CCRPI)
comparison: 2014 to 2017
Meana-Meanb
-3. 5

t
-2.05

df
5

P (one-tailed)
0.04

Note: Data includes all four elementary-level schools, one middle school, and the one high school. 2014 serves
as the baseline and is compared to 2017. Note that in 2018 CCRPI rubrics were changed and cannot becompared to previous years.

Category 3: Developing and Supporting Joint Programs
Four programs that are based on the independence and interdependence of the professional
development school partnership are the Early College program, the Montessori Academy, anda community butterfly house and garden and the newly-established educational doctorate.
The partnership between the District and University includes an Early College program locatedon a university campus (see: https://www.gcsu.edu/earlycollege). This program, which serves
approximately 300 primarily first generation and at-risk middle and high school students,
allows participants to complete their daily Georgia middle and high school education
requirements at the University. High school student participants are allowed to dually enroll at
the University to attend college courses. The Early College Program benefits College of
Education teacher candidates, as well, in that it enables them the opportunity to have fieldplacement in the same location as their education courses, which is important for students
without transportation. The program consistently achieves 100% college placement of highschool seniors.
Another recent example of collaboration between the partners is the creation of a Montessori
Academy at the District's Early Learning Center (see: https://montessorielc.com/). The
Academy serves children aged 6 weeks to Kindergarten, and provides parents access to theMontessori delivery model. Employees of the Academy are certified teachers and trained inMontessori. District and University employees have priority in enrolling their children and,once they are all enrolled, all additional seats are open to students from the community.
A third emerging program involves the District, University and Milledgeville Rotary Club. The
Club will provide funding and volunteer support for a butterfly house and community garden.
The partnership of the Dean of Education and Superintendent, both Rotary Board members,

181

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

demonstrated that there is a strong partnership which showed the Rotary it was an initiative
worth making the investment.
A fourth program that supports the PDS relationship is the new educational doctorate that
supports leaders in the District. Two District principals are already enrolled with more applying
to enter subsequent cohorts. The doctorate was formed to help support current and emerging
leaders in the District so they would not need to drive hours away for a program.
Category 4: Positive Behavior Interventions and Supports
Another important component of the partnership involves collaboration in the area of Positive
Behavior Interventions and Supports (PBIS). Through the partnership’s monthly meetings,
assistance with district-wide behavior management was identified as a need in the District andalso a need for teacher candidates so that they could observe, develop, and implement positive
behavior management approaches. Because PBIS is a statewide initiative in Georgia, theDistrict applied to become a PBIS district. After introducing school District representatives tothe staff of the Georgia DOE’s PBIS team, the DOE’s team agreed to assist the District. The
resulting student behavior support process began during Spring Semester 2015, when the
University assigned a professor with an extensive background as a practitioner in behavior
support to work with the District as a professor-in-residence in its efforts to address behavior
management.
The behavior support process adopted by the PDS partnership was multifaceted, including
components of establishing, training, and supporting the use of PBIS as the District’sframework for managing student behavior. In tandem with this effort, the partnershipestablished a district-level behavior support program and related processes, contributions to apartial reconfiguring of the existing student code of conduct, and behavior management support
program for teachers whose classroom management practices caused administrators concern.
In order to supplement the District’s new proactive approach, the superintendent hired theDistrict’s first behavior specialist, a former social worker with experience in counseling youth
in a local mental health setting. The professor-in-residence met extensively with the new
behavior specialist.
Next, the superintendent requested that the professor-in-residence conduct initial introductory
training sessions pertaining to PBIS and proactive behavior management prior to the advent ofDistrict implementation on the first day of the fall term in 2015. During the summer of 2015
the professor-in-residence planned and conducted professional development training sessions
in PBIS for District administrators, school-based PBIS leadership teams, existing teachers, andnew teachers alongside the District’s superintendent and PBIS coordinator.
The Office Disciplinary Referral (ODR) data tracked throughout the initial year of
implementation reflected an immediate and substantial decline in district-wide office referrals
and a decrease in the District’s suspension rates across each of its seven schools. According to
Georgia Department of Education data, referrals were greatly reduced (see tables 6 & 7). This
trend largely continued throughout the following two school years among each of the District’s
elementary schools with progress fluctuating somewhat at the middle and high school levels.

182

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Table 6: District-wide office disciplinary referral data (2014 to 2016)
SchoolBH Elementary
CS Elementary
ER Elementary
MW Elementary
OH Middle School
BH High School
Total
Mean

2014 Referrals
270
249
734
367
1508
2748
5876
979

2016 Referrals
153
374
578
230
1193
1420
3948
658

Note: Data includes referrals from all six schools in the District. 2017 and beyond could not be compared due tosystematic changes in the data collection; Data source: https://www.gadoe.org/schoolsafetyclimate/Pages/StudentDiscipline.aspx

Table 7: District-wide office disciplinary referral comparison (2014 to 2016)
Meana-Meanb
321

t
+1.53

df
5

P (one-tailed)
0.09
Note: data includes referrals from all six schools in the District. 2017 and beyond could not be compared due tosystematic changes in the data collection.

PBIS challenges. Challenges are expected with the implementation of any PDS initiative, andthis partnership was no exception. Initial challenges reported by the professor-in-residence
included procuring entrée into schools and the cooperation of schools’ administration andfaculty. However, once the superintendent articulated her backing and support of the professor
as an equal partner in PBIS implementation, cooperation among staff increased dramatically,
and the resulting relationships flourished throughout the extent of the faculty member’s
involvement with the District. As an example, at the beginning of the professor’s involvementin Fall 2015 an elementary principal was unresponsive to requests to hang student-friendly
PBIS matrix posters in key locations throughout the building (a common component of PBIS
implementation), arguing that the posters would damage wall paint. The superintendent spoke
privately with the principal, helping him to understand that she backed all PBIS-relatedsuggestions and expected him to work with the professor as her designee in the initiative. Fromthat moment forward, the principal became an active partner-and ultimately an advocate-of
PBIS-related practices such as this one.
Another challenge to the behavior support component of the PDS partnership was one not
atypical in many such cooperative efforts-mission creep. The professor-in-residence, who had
served as an investigator into allegations of discrimination in his previous P-12 directorship,
was asked to conduct a time-consuming formal Human Resources investigation into allegations
that one of the District’s administrators had discriminated against a student with disabilities.
Though the faculty member was willing and able to conduct the investigation, the investigative
process proved to be extensive and detracted somewhat from the time he was able to devote tomonitoring district-wide PBIS implementation.
Limitations of the Study
The primary limitation of the study is that there can be confounding variables such as changes
in the administration and teaching staff, other state, district, and school-wide initiatives, and183

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

community influences. These are often hard to eliminate from something like the CCRPI,
which is an overall score for an entire school.
Recommendations for Further Research
The researchers feel that it is important to go beyond the administrators and key personnel
looking at the program’s success and interview individual teachers to see how they see thepartnership. This may lead to new ways to collaborate in the future. In light of the many
professional development schools that were not able to sustain the relationship, we need tocontinue to explore ways to keep our program strong (Dresden, Blankenship, Capuozzo, Nealy,& Tavernier, 2016).
Because PDS partnerships vary greatly depending upon the contexts and dynamics involved
between participating partners, the authors are hesitant to generalize beyond the present
context. Consistent with the concept of reader generalizability, the reader might best be able
to determine whether our experiences are relevant to their own individual contexts.
In summary, the partnership is an overall success. Literacy continues to improve and behavioral
incidents continue to decline. Early College and the Montessori Academy are both doing well.
Both parties agree that it is important to continue the relationship.
Conclusion
Our PDS model exemplifies how the joint PDS mission has reached beyond each institution’s
ability to provide the experiences and services in isolation. For example, when the Early
College program required space for P-12 students at the University’s campus while, inconverse, the joint Montessori Academy needed space for its pre-kindergarten children within
the District’s Early Learning Center. The interdependence in the PDS allows the partners toaccomplish the overarching joint mission to improve education at all levels in our communityfrom six weeks old to the doctoral level. The University provides grant assistance, professional
development, and courses to K-12 teachers, while the District opens its classrooms for fieldobservation and practice with the University’s teacher candidates.
Since 2015, the Professional Development School partnership between the university and theschool district exemplifies the PDS guiding essentials of achieving “a comprehensive missionthat is broader in its outreach and scope than the mission of any partner and that furthers theeducation profession and its responsibility to advance equity within schools and, by potential
extension, the broader community” (National Association of Professional Development
Schools, 2008b). Though the evolution of the partnership, expected challenges have given way
to a multi-faceted and highly effective delivery of services that have benefited both the
University and the District in a profound way.

184

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

References
Abdal-Haqq, I. (1999). Unraveling the professional development school equity agenda.
Peabody Journal of Education, 74(3/4), 145–160. Retrieved fromhttp://www.jstor.org/stable/1493129
Al Seyabil, F. (2017). Students’ and teachers’ views on school-university partnership in the
Omani EFL context. International Education Studies, 10(3), 125–133.
https://doi.org/ 10.5539/ies.v10n3p125
American Association of Colleges for Teacher Education. (2018a). Colleges of education: anational portrait. Washington, DC: Author.
American Association of Colleges for Teacher Education. (2018b). A pivot toward clinicalpractice, its lexicon, and the renewal of educator preparation. Washington, DC:
Author.
American Montessori Society (n. d.). What is Montessori education? Retrieved fromhttps://amshq.org/About-Montessori/What-Is-Montessori
Andrews, P. G., & Leonard, S. Y. (2018). Reflect, analyze, act, repeat: Creating criticalconsciousness through critical service learning at a professional development school.
Education Sciences, 8(148), 1–13. https://doi.org/10.3390/educsci8030148
Baldwin County School System. (2015). Charter system application. Retrieved fromhttps://www.gadoe.org/External-Affairs-and-Policy/Charter-Schools/Documents/2 Baldwin Petition.pdf
Bentley, E., & Hendricks, M. B. (2018). Dreaming big: The ongoing story of thecollaboration between a university, a school district, an author, and an illustrator.
School-University Partnerships, 11(4), 78–86.
Board of Regents of the University System of Georgia. (2017). Early college. Retrieved fromhttps://www.gaearlycolleges.org/
Boyer, E. L. (1996). The scholarship of engagement. Bulletin of the American Academy ofArts and Sciences, 49(7), 18–33. Retrieved from http://www.jstor.org/stable/3824459
Breault, R. (2014). Power and perspective: the discourse of professional development school
literature. Asia-Pacific Journal of Teacher Education, 42(1), 22–35.
https://doi.org/10.1080/1359866X.2013.869547
Caparo, M. M., Caparo, R. M., & Helfeldt, J. (2010). Do differing types of field experiences
make a difference in teacher candidates’ perceived level of competence? Teacher
Education Quarterly, 37(1), 131–154.
Carpenter, B. D., & Sherretz, C. E. (2012). Professional development school partnerships: An
instrument for teacher leadership. School-University Partnerships, 5(1), 89–101.
Chang, D. F., Chen, S. N., & Chou, W. C. (2017). Investigating the major effect ofprincipal’s change leadership on school teachers’ professional development. IAFOR
Journal of Education, 5(3). https://doi.org/10.22492/ije.5.3.07
Colwell, C., MacIsaac, D., Tichenor, M., Heins, B., & Piechura, K. (2014). District andUniversity perspectives on sustaining professional development schools: Do the
NCATE standards matter? Professional Educator, 38(2), 1–12. Retrieved fromhttps://files.eric.ed.gov/fulltext/EJ1048336.pdf

185

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Cressey, J. M., Whitcomb, S. A., McGilvray-Rivet, S. J., Morrison, R. J., & ShanderReynolds, K. J. (2018). Handling PBIS with care: scaling up to school-wide
implementation. Professional School Counseling, 18(1), 90–99.
https://doi.org/10.1177/2156759X0001800104
Creswell, J. W., & Plano Clark, V. L. (2006). Designing and conducting mixed methods
research. Thousand Oaks, CA: Sage.
Darling-Hammond, L. (1994). Professional development schools: schools for developing aprofession. New York: Teachers College Press.
Darling-Hammond, L. (1998). Teacher learning that supports student learning. Educational
Leadership, 55(5), 6–11.
Darling-Hammond, L. (2010). Teacher education and the American future. Journal of
Teacher Education, 61(1-2), 35–47. https://doi.org/10.1177/0022487109348024
Darling-Hammond, L., & Bransford, J. (Eds.). (2005). Preparing teachers for a changing
world: What teachers should learn and be able to do. San Francisco: Jossey-Bass.
Darling-Hammond, L., Ramos-Beban, N., Altamirano, R. P., & Hyler, M. E. (1998). Be thechange. Reinventing school for student success. New York: Teachers College Press.
Denzin, N. K., & Lincoln, Y. S. (2011). The SAGE handbook of qualitative research.
Thousand Oaks, CA: Sage.
Dresden, J., Blankenship, S. S., Capuozzo, R. M., Nealy, A. U., & Tavernier, M. D. (2016).
What is a PDS? Reframing the Conversation. School-University Partnerships 9(3),64–80. Retrieved from http://napds.org/wp-content/uploads/2016/10/93-all.pdf
Georgia College & State University & The Baldwin County School District. (2015).
Memorandum of understanding. Milledgeville, GA: Authors.
Georgia College & State University. (2016). Our path to preeminence: The 2016-2021
Georgia College strategic plan. Retrieved from https://www.gcsu.edu/
Georgia Department of Education. (2014). Georgia positive behavior interventions andsupports (PBIS) strategic plan 2014-2020. Atlanta, GA: Author. Retrieved fromhttps://www.gadoe.org
Georgia Department of Education. (2019). A family’s guide to Georgia’s College and Career
Ready Performance Index. Atlanta, GA: Author. Retrieved fromhttps://www.gadoe.org
Georgia Professional Standards Commission. (2019). Positive behavior intervention andsupports (PBIS) endorsement program. Atlanta, GA: Author. Retrieved fromhttps://www.gapsc.com/Rules/Current/EducatorPreparation/505-3-.109.pdf
Goodlad, J. (1990). Teachers for our nation's schools. San Francisco: Jossey-Bass.
Goodlad, J. (1994). Educational renewal: better teachers, better schools. San Francisco:
Jossey-Bass.
Gordon, G., & Morse, E. V. (1975). Evaluation research. Annual Review of Sociology, 1,
339–361.
Hirsch, S. E., Brown, C. P., Halley, E. A., Beach, J., MacSuga-Gage, A. S., Spence, S. E., &Kennedy, M. (2019). Ten tips to consider when creating and screening PBIS films.
Beyond Behavior, 28(2), 79–89. https://doi.org/10.1177/1074295619845830

186

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Holmes Group. (1986). Tomorrow’s teachers: a report of the Holmes Group. East Lansing,
MI: Holmes Group. Retrieved from https://files.eric.ed.gov/fulltext/ED270454.pdf
Holmes Group. (1990). Tomorrow's schools: principles for the design of professional
development schools. East Lansing, MI: Holmes Group.
Horner, R. H., Kincaid, D., Sugai, G, Lewis, T. Eber, L., Barrett, S., & Johnson, N. (2014).
Scaling up school-wide positive behavioral interventions and supports: experiences ofseven states with documented success. Journal of Positive Behavior Interventions,16(4), 197–208. https://doi.org/10.1177/1098300713503685
Horner, R. H., Sugai, G., & Lewis, T. (2015). Is school-wide positive behavior support anevidence-based practice? Retrieved from https://www.pbis.org/research
Johnson, B. R., Onwuegbuzie, A. J., Turner, L. A. (2007). Toward a definition of mixed
methods research. Journal of Mixed Methods Research, 1, 112–133.
https://doi.org/10.1177/1558689806298224
Jupp, V. (Ed). (2009). The Sage dictionary of social research methods. Los Angeles, CA:
Sage.
Levine, M. (1992). Professional practice schools: linking teacher education and school
reform. New York: Teachers College Press.
Levine, M. (1998). Designing standards that work for professional development
schools. Washington DC: National Council for the Accreditation of Teacher
Education.
Lewis, S., & Walser, T. M. (2016). Advancing a professional development system: evolution
and evaluation. School-University Partnerships, 9(3), 130–152.
Lowry, R. (2019). Concepts and applications of inferential statistics. Retrieved fromhttp://vassarstats.net/textbook/index.html
Montessori Academy at the Early Learning Center (n. d.). About. Retrieved fromhttps://montessorielc.com/about
Murrell, P. C. (1998). Like stone soup: The role of the professional development school in the
renewal of urban schools. New York: AACTE Publications.
National Association for Professional Development Schools. (2019). Vision, purpose, history.
Retrieved from https://napds.org/visionpurposehistory/
National Association for Professional Development Schools. (2008a). 9 essentials. Retrieved
from https://napds.org/nine-essentials/
National Association for Professional Development Schools. (2008b). What it means to be aprofessional development school. Retrieved from https://napds.org/wpcontent/uploads/2014/10/Nine-Essentials.pdf
National Association for Professional Development Schools (n.d.). The nine required
essentials of a PDS. Retrieved from https://napds.org/nine-essentials/
Netzel, D. M, & Eber, L. (2003). Shifting from reactive to proactive discipline in an urban
school district. Journal of Positive Behavior Interactions, 5(2), 71–79.
https://doi.org/10.1177/10983007030050020201
Newman, W. L. (2011). Social research methods: qualitative and quantitative approaches(7th ed.). Boston, MA: Allyn & Bacon.

187

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Nzarirwehi, J., & Atuhumuze, F. (2019). In-service teacher training and professional
development of primary school teachers in Uganda. IAFOR Journal of
Education, 7(1). https://doi.org/10.22492/ije.7.1.02
Parker, A. K., Parsons, S. A., Groth, L. A., & Brown, E. L. (2016). Pathways to partnership: adevelopmental framework for building PDS relationships. School-University
Partnerships, 9(3), 34–48.
Parsons, B., & Renyi, J. (1999). Breathing the professional development school spirit into allschools. Peabody Journal of Education, 74(3/4), 263–276. Retrieved fromhttp://www.jstor.org/stable/1493137
Polly, D., Reinke, L. T., & Putnam, S. M. (2019). Examining school-university partnership:
synthesizing the work of Goodlad, AACTE, and NAPDS. School-University
Partnerships, 12(3), 24–40.
Renyi, J. (1996). Teachers take charge of their learning. Transforming professional
development for student success. Washington, DC: The National Foundation for the
Improvement of Education.
Richey, R. C., & Klein, J. D. (2007). Design and development research. Mahwah, NJ:
Lawrence Erlbaum Associates.
Ronen, I. (2018). The impact of organizational and personnel changes on a college-school
partnership vision. Teacher Development, 22(5), 668–684.
https://doi.org/10.1080/13664530.2018.1473288
Schack, G. D. (2011). Multiple roles and relationships of professional development school
liaisons: the struggle for collegiality within hierarchical settings. Peabody Journal of
Education, 74(3/4), 306–309. Retrieved from https://www.jstor.org/stable/1493142
Sugai, G, & Horner, R. H. (2009). Responsiveness-to-intervention and school-wide positive
behavioral supports: integration of multi-tiered system approaches. Exceptionality, 17,
223–237. https://doi.org/10.1080/09362830903235375
Teitel, L. (2004). Two decades of professional development school development in the
United States. What have we learned? Where do we go from here? Journal of Inservice Education, 30(3), 401–416. https://doi.org/ 10.1080/13674580400200335
Tracz, S. M., Beare, P., & Torgerson, C. (2018). A longitudinal case study of a schooluniversity partnership for training teachers. The Journal of School Administration
Research and Development, 3(1), 42–56.
Walsh, M. E., & Backe, S. (2013). School–university partnerships: reflections andopportunities, Peabody Journal of Education, 88(5), 594–607,
https://doi.org/10.1080/0161956X.2013.835158
Corresponding author: Robert Sumowski
Contact email: Robert.sumowski@gcsu.edu

188

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Acronyms
AACTE: American Association of Colleges for Teacher Education
COE: College of Education
CCRPI: College and Career Readiness Performance Index
DOE or GaDOE: Georgia Department of Education
GCSU: Georgia College & State University
MOU: Memorandum of Understanding
NAPDS: National Association for Professional Development Schools
ODR: Office Disciplinary Referral
PBIS: Positive Behavior Interventions and Supports
PDS: Professional Development School
SW-PBIS: School-wide Positive Behavioral Interventions and Supports
STEM: Science, Technology, Engineering, and Mathematics
STEAM: Science, Technology, Engineering, Arts, and Mathematics

189

IAFOR Journal of Education

Volume 7 – Issue 2 – Winter 2019

Reviewers for Volume 7 – Issue 2
The editorial team would like to thank the following reviewers for their contributions to peer
review for this issue of the journal. Their dedication and assistance are greatly appreciated.
Jose BarlisSantosh Behera
Christina Belcher
Angelina Borican
Ivy Chia
Shiao-wei Chu
Caroline Dacwag
Denise de Souza
Alaster Douglas
Tania Fonseca
William Frick
Beena Giridharan
Alice Schmidt Hanbidge
Elaine Huber
Habsah Hussin
Leda Kamenopoulou
Intakhab Khan
Lynda Leavitt
Jin LeeAndrew Leichsenring
Thelma Manansala
Teri Marcos

Catalino N. Mendoza
Alyson Miller
Lena Mishieva
Massoud Moslehpour
Suja Nair
Nomazulu Ngozwana
Anuli Njoku
Agnes Papadopoulou
Elena Raevskikh
Cornelius Riordan
Ilana Ronen
Bethe Schoenfeld
Raimond Selke
Meliha Simsek
Pearl Subban
Consorcia Tan
Anna Toom
Daniel Velasco
Kyle WhitfieldYoshihiko Yamamoto
Tony Yeigh
Huanshu Yuan

190

the iafor

journal of education

ISSN: 2187-0594