I
S
S
N:
21870594

The IAFOR Journal of Education	
  Volume 1 - Issue 1 - Spring 2013

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  

iafor
The IAFOR Journal of Education
Volume 1 – Issue 1 – Spring 2013

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

IAFOR Publications
Executive Editor:
Joseph Haldane
The International Academic ForumThe IAFOR Journal of Education
Editor:Sandra Kroh, University of Pikeville, USA
Associate Editors:
Bernard Montoneri, Providence University, Taiwan
Martha J. Payne, Michigan State University, USA

Published by The International Academic Forum (IAFOR), Japan
IAFOR Publications. Sakae 1-16-26-201, Naka-ward, Aichi, Japan 460-0008
Executive Editor, IAFOR Publications: Joseph Haldane
Design: Thomas Haldane. Editorial Assistance: Kiyoshi Mana, Melissa Choi
The IAFOR Journal of Education
Volume 1 – Issue 1 – Spring 2013
IAFOR Publications © Copyright 2013
ISSN: 2187-0594 (Online) http://iafor.org/educationjournal.html

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Notes on Contributors

Magdalene Meow Khee Chew is a senior lecturer in the School of Computing and IT at
Taylor’s University, Malaysia where she has been teaching courses related to English Language
and Computer Ethics since 2007. She completed her Master of Education (with distinction) andher undergraduate studies at the University of Malaya, Malaysia. She is a writer of educational
materials for the teaching and learning of English Language and Literature in English. To date,
Magdalene has co-authored four secondary school English Language textbooks approved by the
Malaysian Ministry of Education. Her research interests include Technology-enhanced
Language Learning (TELL), blended learning and approaches to teaching and learning in higher
education.
Scott Thompson-Whiteside is currently Acting Dean of the Faculty of Design at Swinburne
University of Technology in Melbourne, Australia. Since 2005, Scott has been on the Faculty
Executive as either Associate Dean International, a role he retains, or Deputy Dean. He has been
responsible for a range of international activities within the faculty from recruitment, exchange,
study tours, transnational education and internationalisation of the curriculum.
Scott has extensive experience in teaching and managing programs, departments and schools in
the UK, Malaysia, and Australia. He currently teaches design management within the faculty
and in 2006 won the Vice-Chancellor's teaching award for higher education.
Scott’s PhD is from The University of Melbourne, Australia at the Centre for the Study ofHigher Education. His PhD and research explores how the massification and internationalisationof higher education has changed our notion of how universities set, monitor and assess academic
standards. His interests lie in higher education policy and management, internationalisation,transnational education and quality assurance.

Catherine Lee Cheng Kiat, an academic in linguistics and Teaching English as a Second
Language, embarked on her teaching career as an English teacher upon graduation from a
Teacher’s Training College. She holds a Bachelor of Arts (Hons) in English Language
Teaching, with special reference to ESP from the University of Birmingham, United Kingdom.
Thereafter, she was attached to a Malaysian Polytechnic in Jitra, Kedah serving as an English
lecturer in the General Studies Department. During her eight years there, she helped to develop
Teaching and Learning Modules for Technical and Business English.
She obtained her Masters in English as a Second Language (MESL) from the University of
Malaya, Malaysia and joined Taylor’s University, School of Communication, Malaysia in 2005,
after more than twenty years of service in the Education system. Her career at Taylor’s started
out as a Stream Coordinator for English Language in her faculty and she is also the Programme

1

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Director for the Bachelor of Arts in Communication and Media Management, University ofSouth Australia, Australia.
Her passion in teaching brings her to explore different platforms for a variety of classroom
teaching. Her areas of interest in research include the use of blogs, social media platforms andenhancing learner autonomy in teaching and learning. She’s currently involved in the pilot
project for blended learning in the School of Communication, whereby, the use of social media
platform is embedded in her teaching schedule to promote active participation for online
discussion.

Yoshihiko Oya
Brief Personal History:
1979 Received Master of Science from Nagoya Univ.
1983 Lecturer of Nagoya Univ. of Arts and Sciences Junior College
1993 Associate professor of Nagoya Univ. of Arts and Sciences Junior College
2001 Professor of Nagoya Univ. of Foreign Studies
Specialty:
Informational Education
Affiliated Society:Japan Society for Educational Technology
Kimiko Uchida
Brief Personal History:
1999 Lecturer of Nagoya Univ. of Arts and Sciences Junior College
2007 Associate professor of Nagoya Univ. of Arts and Sciences Junior College2010 Received Master of Education from the Aichi University of Education2011 Associate professor of Nagoya Univ. of Arts and Sciences
Specialty:
Informational Education
Affiliated Society:Japan Society for Educational Technology Information Processing Society of Japan

Chai Meenorngwar graduated from Khon Kaen University in 1992. For several years, heworked as a lecturer in the Information Technology Department at Valaya Alongkorn Rajabhat
University under the Royal Patronage, Pathum Thani, Thailand. He graduated from the Asian
Institute of Technology in 1997 with a Master’s degree in Computer Science. He also completed
a Master of Philosophy in Computer Science from The University of Liverpool in 2010.
His research area is in ontology development methodology, the subject of which he has
published several papers in. He continues to develop the Semantic Web as a programmer. He is a
professional in web database and programming. Therefore, he not only works on themethodology but also implements the solutions.

2The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Journal of Education Editors

Editor: Dr Sandra Kroh
Ph.D. (University of Pikeville)
Sandy Kroh received her PhD in Applied Linguistics from Ball State University in the UnitedStates. Her fields of study are Second Language Acquisition, Teacher Training and Vygotsky’s
Sociocultural Theory of Learning. She has worked for the Ministry of Education in the People’s
Republic of China where she helped rewrite the Junior English for China textbooks and haspresented and published in countries and journals all over the world.
Dr. Kroh is currently an Associate Professor at University of Pikeville, USA where she is the
director of Global Education, which includes branch campuses in China. She speaks Mandarin
and German. She collects fairy tales from around the world and is researching the Appalachiandialect in the Commonwealth of Kentucky.

Associate Editor: Dr Bernard Montoneri
Ph.D. (Providence University)
Bernard Montoneri earned his Ph.D. (African, Arab, and Asian Words; History, Languages,
Literature) and his BA in Chinese from the University of Provence, Aix-Marseille I, France. He
is currently an Associate Professor at Providence University, Taichung, Taiwan. He teaches
Literature (European, Children, and British) and languages (French, English, and Italian).
Bernard has published about 30 journal and conference papers, 5 books and has obtained 16
teaching or research projects. His research interests include European literature, children
literature, English writing, automated scoring systems, teaching and learning evaluation, dataenvelopment analysis, and teaching methods. He is a reviewer for top academic journals such as
Review of Educational Research (impact factor: 3.169; rank: 3 out of 203 in Education &Educational Research), American Educational Research Journal (2.393), Teaching and Teacher
Education (1.322), and European journal of Operational Research (1.815).

3

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  

4

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Table of Contents

Notes on Contributors

p. 1

Journal Editors

p. 3

Table of Contents

p. 5

Introduction
Sandra Kroh

p. 6

Are Standards-based Quality Systems a Threat to theInternationalization of Teaching and Learning?
Scott Thompson-Whiteside
A Methodology to Develop Ontologies for Emerging Domains
Chai Meenorngwar
Evaluation of Teaching Performance of English Courses by Applying
Data Envelopment Analysis and Two-phase Segmentation
Bernard Montoneri
Practical Consideration of Pair Problem Solving in Computer Literacy
Education
Yoshihiko Oya
Kimiko Uchida
Reflections on the Final Year Learning Experience - Designing a
Capstone Experience
Keith Thomas
Kin Chi Wong
Yi Ching Li
Ching Yan Hung
Using a Blog to Facilitate Extensive Reading: An Exploratory Study
Magdalene Meow Khee Chew
Catherine Cheng Kiat Lee5

pp. 11

pp. 39

pp. 79pp. 103

pp. 123

pp. 149

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Introduction

The understatement of the 21st century is that technology has changed our world.
Education is scrambling to keep up with its students’ use of technology and trying to“think outside the box” (which is becoming a normal thought process as well),concerning how to utilize technology for educational purposes. This inaugural issue ofthe IAFOR Journal of Education is dedicated to developing new methodologies andstandards needed due to technology as well as how technology is being used in theclassroom. The articles included reflect upon the theme of the Asian Conference on
Education of “Learning and Teaching in a Globalized World”. They address manydifferent disciplines as well as explore how these different disciplines function
technologically as well as globally.

Since globalization has a direct connection toeducation, especially as technology has “shrunk” the size of the world, we do not think
twice about referring to education as a global concept, rather than as an entity thatbelongs to a specific country. Education belongs to the world.

Thompson-Whiteside explores what standard based quality systems mean for the
regulation of transnational education, particularly as it pertains to international
partnerships with Australian universities.

Universities collaborate world-wide andquality control is an issue. With different governing bodies for different institutions, howregulation is implemented has global implications. Regulation of transnational education
will need people of vision.

6

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Meenorngwar proposes a methodology that helps to make the insurmountable amount of
information available on the internet easier to access. The globalization of knowledge
management has become unwieldy and researchers often find it difficult to find theproper domains for their searches. Meenorngwar proposes a methodology for developing
ontologies for emerging domains.

Montoneri discusses the analysis of teacher performance, and how that analysis canimprove the learning environment. Within the teacher evaluations, his analysis focuses
on four indicators, two input, and two output. Accordingly, the results give teachers
suggestions for improving performance from communicating grades clearly to helping
students’ learning performances. Even though his study focused on English as a foreign
language, this analysis could be applied to any subject, depending on the indicators
selected.

While Montoneri focuses on teacher performance, Oya and Uchida look at student
performance in a Computer Literacy course. They state that students’ performance in thecourse improves if they work in pairs, as opposed to working individually. Their side
note in the study is quite interesting in that their study revealed the most effective pairingof students were the pairs who exhibited, “… a small difference in basic academic ability,
a large difference in PC experience, and a partner of the opposite sex”.

Thomas et al. discuss curriculum development in Hong Kong universities, specifically, acapstone experience for students’ final year project. They took a qualitative approach to7

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

their study in order to gain insight from personal interviews and conversations from theirstudent subjects.

Through these insights, they discovered that the present capstone

experience addressed closure of the university experience as well as reflection on theirlearning. However, it did not address integration of their learning and transition into theworkforce. Thomas et al. discovered that curriculum design had a direct correlation to
the effectiveness of a capstone experience.

Chew and Lee’s article focuses on the specific course of Extensive Reading and how tomake it interesting for students of the digital generation. They suggest that a gap exists
for this generation between the texts they read outside the class and traditional texts usedwithin the classroom. They propose using a reading blog to facilitate the traditional
Extensive Reading methodology. Their study showed that their subjects unanimously
agreed that the blog improved their motivation concerning reading.

An interestingfinding was that 86% of the participants enjoyed the social aspect of the blog. Oftentimes,
sitting in front of a computer is not considered a social event. However, the students feltthat reading a blog, and material that others posted, was more social than reading a textthat the teacher had chosen. This finding begs the redefinition of a social event.

It has been a supreme pleasure in editing this journal. All articles were interesting and it
was not an easy decision to narrow it down to those that were included. I applaud all theauthors who submitted their work for inclusion and encourage them to keep researching
and writing. To those who made the final cut, I thank them for their patience in working
with us as this journal has taken much longer to publish than anticipated. The editors and

8The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

I have learned much, and look forward to publishing the next edition in a much quicker
fashion.
*A final note on the type of English contained within this journal. Since the Asian
Conference on Education brings together presenters with many world Englishes, a
conscious effort has been made to keep the “English voice” of the author of each article
intact. Therefore, when you read the articles, you may hear an Australian, British, or
Malaysian English voice. This is in keeping with the spirit of global education. The
digital voice must be global as well.

9The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  

10

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Are Standards-based Quality Systems a Threat to the Internationalization ofTeaching and Learning?

Scott Thompson-Whiteside

	
  

11

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Abstract
This paper explores the current shift in Australia’s higher education system moving toa more explicit, standards-based quality system and its potential impact oninternational partnerships in teaching and learning, particularly in Asia. The new
Tertiary Education Quality and Standards Agency and the underlying Higher
Education Standards Framework have the potential to threaten a large number oftransnational or cross-border programs delivered outside of Australia. With over one
hundred and fifty thousand tertiary students studying Australian programs in Asia, theimpact could be significant. It would also be significant for countries that leverage ofAustralian Universities to build human capacity within their country. The paper
highlights the current practice of assuring equivalent and comparable academic
standards in transnational education and explores how shifting to a more precise
standards framework will require more explicit demonstration of standards acrossteaching, learning and student outcomes. If equivalent or comparable standards were
to be achieved across the whole standards framework, it is likely to constrain theopportunities for internationalization and the formation of new transnational
partnerships.

Keywords: tertiary education quality, standards framework, transnational

	
  

12

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Introduction
Australia’s higher education system is undergoing considerable change. Since
publication of the Review of Australian Higher Education (Bradley et al., 2008),
otherwise known as the Bradley Review, there has been increasing emphasis anddebate on the notion of standards in higher education. The review stated that,“Australia must enhance its capacity to demonstrate outcomes and appropriate
standards in higher education if it is to remain internationally competitive andimplement a demand driven funding model” (p.128). The review also recommended aneed for clarification and agreed measurements of standards and for institutions todemonstrate their processes for setting, monitoring and maintaining standards. In
essence there was seen to be a need for institutions to explicitly demonstrate theirstandards for the sake of public accountability. As a consequence of the Bradley
Review, the Tertiary Education Quality and Standards Agency (TEQSA) was
legislated in March 2011 and established in July 2011 with responsibility for
implementing a new Higher Education Standards Framework. This framework has
five components and aims to specify more precisely the standards expected frominstitutions. Institutions are expected to demonstrate achievements against those
expectations.

The more precise nature of the standards framework, in particular the teaching andlearning component of the framework, will require institutions to demonstrate a whole
range of teaching and learning standards. These standards will be assessed and judged
in a number of ways, using both qualitative and quantitative indicators. The precise
criteria for assessing teaching and learning standards has yet to be fully defined but
TEQSA’s decision to move away from institutional audits (Lane, 2011) suggests that

	
  
13

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

more emphasis will be placed on a range of quantitative data and benchmarked
against institutional and national expectations.

The standards of teaching and the standards of students’ learning will obviously focus
on teachers and students in Australia.

However, what has yet to be publically

discussed is that it will also affect teachers and students who teach or study inAustralian programs outside of Australia. These are students studying in Australiantransnational programs. With nearly one hundred thousand students studying inAustralian higher education in transnational programs (plus a further fifty thousand
vocational education students), the need to demonstrate precise measures of teaching
and learning standards may have considerable ramifications. If the current policy
continues to mandate equivalent or comparable standards, a more precise, standardsbased quality system may restrict the ability for Australian institutions to engage intransnational partnerships. It may also constrain the types of partnerships and the way
in which curriculum, teaching and assessment is done.

This paper provides some background to the current regulation of transnational
education and in particular the notion of equivalent and comparable standards. It willthen address the new Higher Education Standards Framework and explore theimplications for Australian transnational education.

Australian Transnational Education
The growth of transnational education, also known as cross-border education, sincethe 1990s has coincided with the growing demand for internationally recognised
qualifications, the globalisation of professions and changing socio-economic

	
  
14

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

circumstances in Asia (McBurnie and Ziguras, 2007). Australia has been well
positioned to tap into this growth. While many students choose to travel to Australia
to study, many stay in their home country, or travel to a third country to enrol in an
Australian program.

Some of these students may be studying at an Australianoffshore campus, and some may be enrolled at an institution that is in partnership with
an Australian institution. In either case, transnational students are typically enrolled in
an Australian program and upon successful completion will receive an award from theAustralian institution. For the purposes of this paper I will use UNESCO’s definitionof transnational education as,…all types and modes of delivery of higher education study
programs, or sets of courses of study, or educational services (including
those of distance education) in which the learners are located in acountry different from the one where the awarding institution is based.
Such programs may belong to the education system of a State different
from the State in which it operates, or may operate independently of any
national education system (UNESCO, 2001, p.2).

In 2009 Australian Universities were offering 889 transnational programs delivered
outside of Australia with the majority of programs based in Singapore, Malaysia,
China, Hong Kong and Vietnam (Universities Australia, 2009). The nationality ofstudents enrolled in those programs also followed the same pattern of countries (AEI,2010). This means that the majority of students studying Australian transnational
were based in their own country of nationality. Currently, Australian higher education
enrolls over 100,000 students in transnational programs and is forecast to reach over

	
  
15

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

400,000 by 2025 (Bohm et al., 2002). With such a significant number of students, theregulation of quality and standards is critical.

The pursuit of transnational partnerships in the 1990s was largely for commercial
reasons. Partnerships were established with little understanding of the risks involved
and with little regulatory or legal framework (McBurnie and Ziguras, 2007).
Currently, the risks and benefits of transnational education are more widely known
and it is recognized that institutions need to be more strategic in their approach todeveloping new transnational partnerships (Connolly and Garton, 2007). Since the1990s there has been significant development in the quality assurance of transnational
programs and cross-border regulation. There are a range of national and international
protocols, guidelines and codes of practice, but because they span different
sovereignties, they are often voluntary.

The regulation of Transnational Education
Transnational education crosses social and cultural boundaries as well as the more
obvious geographical and national boundaries of sovereignty. Students in Australiantransnational programs are both national and international in relation to the hostcountry of study, but few are Australian. Most of the academic staff teaching the
programs are unlikely to be Australian. Students, institutions and staff are boundacross, and sometimes between, different national regulatory frameworks, protocols
and codes of practice. As a result, transnational education creates complex anddynamic tensions in the assurance and demonstration of quality and standards. Thesetensions vary between the host and awarding country depending on the mix ofstakeholders and development of each regulatory system (Verbik and Jokivirta, 2005).

	
  
16

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Different regulatory systems assert different levels of control over the assurance ofstandards in their home country or upon their home-based institutions.

Over time, there has been greater recognition of different regulatory systems and adrive towards the mutual recognition of national quality assurance and regulatory
systems. In turn this has driven the development of common or similar regulatory
systems.

The

internationalizationofhighereducation,

and

with

it

the

internationalization of quality assurance, has had an isomorphic effect on national
quality regulatory systems (Van der Wende, 1999, McBurnie and Ziguras, 2007).
Supranational agencies like the International Network of Quality Assurance Agencies
in Higher Education (INQAAHE) and the European Network for Quality Assurance(ENQA) have emerged. While they are sharing best practice and developing quality
assurance guidelines there is a sense that these supranational agencies are also driving
a convergence of quality systems and a shared understanding of standards.

Nevertheless, these isomorphic effects also have the potential to create conflict.
Regulatory systems are generally national in their scope and are designed to protect
national interests. For transnational education, different stakeholders have different
views. Some have even considered transnational education a threat to national
standards. As Adam (2001) states,
Significant numbers of institutions view transnational education
as some sort of threat to standards and their existence. The scale andintensity of the threat is misjudged as it is currently confined to certain
sectors of educational provision. However, its rapid expansion is likely
to continue unabated and so will its impact. It needs to be subject to

	
  
17

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

appropriate quality control mechanisms before the problems intensify.
Governments and institutions in importing countries must consider why
their students choose imported education. Fear of transnational education
should not translate into ineffective protectionism (p.47).

The general response to the growth of transnational education in the 1990s was forhost countries to increase the regulation of foreign providers or partnerships withforeign awards. However, strategies of tight regulatory protectionism had to bebalanced with trade liberalization to ensure that the host country continued to attract
high quality foreign institutions. This was a difficult balancing act and so it became
apparent that the best way to protect and uphold standards was to have tighter
regulation for institutions who award the qualifications (Harvey, 2004, Knight, 2005).
In other words, the Australian regulation of standards took precedence over any
regulation of a country in which it was being delivered. This does not negate the needfor host country regulation but ultimately the awarding institution is more likely to
pay attention to their home regulatory system.

Australian Protocols and transnational standards
In Australia, the development of a robust quality assurance and regulatory system has
been acknowledged as a critical factor in its success of transnational education(AVCC, 2005a). Whilst the quality assurance of transnational education has largely
been dealt with at an institutional level, the institutions are governed by a national
regulatory system. Through the National Protocols of Higher Education Approval
Processes, Codes of Practice, the Educational Services for Overseas Students (ESOS)
Act, and the work of the Australian Universities Quality Agency (AUQA), Australia

	
  
18

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

has been able to develop a transnational quality framework that is considered bestpractice (Ilieva and Goh, 2010).

In particular, it is the National Protocols of Higher Education Approval Processes,
which provided the initial settings for transnational education. Protocol section 4.2
stateed that if a program is delivered in an offshore campus operated by the Australian
university, “standards should be equivalent” to those in Australia. Alternatively, if a
program is delivered with a third party provider offshore, “standards should becomparable” to those delivered in Australia (DETYA, 2002).

The regulation of Australian transnational education reveals the complexity andambiguity of standards in higher education. There is no explicit description within the
Protocols as to what types of standard it is referring. Nor is there any explicit
information about the definition or level of tolerance within the notion of equivalenceor comparability. This ambiguity raises further questions about who sets, maintains,
and assesses standards since it assumes that the standards in Australia are appropriate
to be delivered in another country.

In April 2005, the Australian Vice Chancellors Committee (now known as
Universities Australia) developed a Code of Practice for the provision of international
students, which included guidelines for transnational education. The guidelines
suggested use of comparability rather than equivalence, broadly following the
UNESCO and OECD codes of practice developed in the same year. The AVCC codesuggested that, “the quality of academic provision and academic support services
offered under the arrangement are comparable” (AVCC, 2005b, p.5). Comparability

	
  
19

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

is tied directly to academic provision and academic support services.

At the same time as the publication of the AVCC Code of Practice, the Australian
government published a discussion paper titled A National Quality Strategy forAustralian Transnational Education and Training (DEST, 2005). Whilst the paper
highlighted the success of Australian transnational programs, it also raised concerns
over the transparency of Australian and institutional quality assurance, accountability
and questioned the equivalence of courses/programs. In May 2005, the AVCC
responded to the discussion paper, suggesting that the government failed to recognize
existing quality assurance measures and requested clearer definitions of ‘equivalent
standards’.
A key element of the discussion paper is that qualifications
obtained offshore are equivalent to those delivered onshore in Australia.
This idea of equivalence needs to be appropriately defined. Australian
universities already address the need for equivalence between onshore
and offshore courses through adherence to Protocol 4.2. The university
interpretation of this protocol is that the equivalence is between
programs offered by the same institution. The Department of Education,
Science and Training needs to confirm that its interpretation ofequivalence, for the purposes of this paper, is equivalence between
programs offered by the same institution” (AVCC, 2005a, p.7).

In this instance, the AVCC was suggesting that equivalent standards were represented
by the fact the programs/curriculum were equivalent and therefore complied to thesame quality assurance mechanisms.

	
  
20

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

By November 2005 an agreed Transnational Quality Strategy was published which
provided a framework for the planning and implementation of programs offshore(AEI, 2005). The Transnational Quality Strategy focused on three areas:• Better communication and promotion of Australia’s quality assurance
systems.
• Improved data collection to inform future strategies.
• A strengthened quality framework that protects and promotes the qualityof Australian transnational education.

The publication did not respond directly to AVCC’s concern of defining equivalency
but was more explicit on the issue. “Courses/programs delivered within Australia andtransnationally should be equivalent in the standard of delivery and outcomes of thecourse, as determined under nationally recognized quality assurance arrangements”(p.1). Without any significant debate, the notion of equivalent standards shifted fromcourses/programs in May 2005, to the delivery and outcomes of the courses/programsby November 2005.

The broad policy statements that developed over 2005 gave significant room forinterpretation and ambiguity. Between the National Protocols and the Transnational
Quality Strategy there was no clear policy as to what types of standards needed to beequivalent or comparable and how they should be measured. There seemed to be noreal understanding of where these different types of standards sit on a spectrum
between equivalency and comparability. The confusion was highlighted in October
2006 in a government commissioned report summarizing a study of fifteentransnational programs in Australian institutions (IEAA, 2006). The report

	
  
21

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013highlighted poor understanding and definitions of terms such as ‘equivalence’,‘comparable’, ‘benchmarks’, or ‘standards’ and recognized that terms are often usedinterchangeably. It went further to suggest that quality assurance in transnational
education was a core concern for all stakeholders, and there was a lack ofunderstanding of how the processes of quality assurance effectively worked with adiverse range of transnational programs and partnerships to ensure standards weremaintained.

Equivalency and comparability of standards
Equivalency and comparability of standards are central components of the Australian
regulatory system for transnational education, however, it is difficult to ascertain
whether these concepts refer to programs, teaching, learning outcomes, student
support and/or experiences. The national Transnational Quality Strategy suggests thatdelivery and outcomes should be equivalent or comparable depending on whether it isan Australian campus or a partnership (AEI, 2005). Not only is there a need forclarification on what the essential anchor points are for demonstrating standards, butalso there is also a need for understanding the acceptable tolerance within equivalentand comparable standards.

Research on the interpretations of equivalence and comparability across a sample ofeighty-five participants within Australian transnational partnerships revealed thatthese terms were used in a variety of ways. “Comparability was generally used tosignify similarity (e.g. It is not of equal standard but is not far off) whereas
equivalence was used to indicate equality or sameness (e.g. It is of same standard)”(Sanderson et al., 2010, p.3). The research suggested that the terms equivalency and

	
  
22

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

comparability were used in reference to standards, programs, assessment, student
experiences and learning outcomes. The activities of assessment were used mostfrequently when questioned about standards in transnational education. Thus, theprocesses of assessment were considered the most valid and reliable reference points
for assuring and demonstrating standards. This supports the view that assessment andthe moderation of assessment in transnational education is the most effective way todemonstrate the standards of graduates (Thompson-Whiteside, 2011a). Moderation
allows for informed judgments and a contextualization of standards.

Considering the variety of delivery models in transnational education, it is difficult tosuggest that any standards could be equivalent considering that the students aredifferent, the lecturers are different, the resources and learning environments aredifferent, and the social and cultural surrounding are different. I suggest the wordingof equivalent standards in transnational education is a misnomer.

Also implicit within the notion of equivalent standards is that one standard is higher
or better than the other. Presumably in this instance, the National Protocols imply that
the Australian standards are superior to offshore ones. The notion of equivalency andthe assumption that Australian campuses are superior to their offshore ones fails torecognize the complexities of transnational education and ultimately is unproductivein generating mutually beneficial, long-term, sustainable partnerships. Since good
partnerships are critical to the success of transnational education (Heffernan, 2005)
the notion of comparability, rather than equivalence, provides a more appropriate
framework of mutual respect and an appropriate level of flexibility. “The use ofcomparability recognises the extent of engagement of importing countries in the

	
  
23

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013transnational endeavor. This goes some way to constructing transnational education as
a mutually productive and reciprocal engagement” (AEI, 2008, p.13). However, it isalso acknowledged that comparability leaves open the potential for too muchinterpretation and needs to be constrained.

The use of comparable standards, rather than equivalent standards, also allows forcontextualization of curriculum and teaching which is seen to positively meet the
specific needs of a diverse group of learners and good teaching practice (Leask,2007). The UNESCO/OECD Guidelines support the view that institutions are to“ensure that the programs they deliver across borders and in their home country are ofcomparable quality and that they also take into account the cultural and linguistic
sensitivities of the receiving country” (UNESCO, 2005, p.15). It suggests that thecontextualization of curriculum and teaching and learning practices are pedagogicallyand culturally appropriate. This, in turn, creates a range of tensions because if thecurriculum or teaching is not equivalent or similar, is it possible to demonstrateequivalent or comparable standards? The presumption is that because the curriculum
content is not the same, it is inferior. As Woodhouse and Carroll note, “Higher
education is a construct in which the method of delivery, which is heavily influenced
by its context, is inseparable from the quality of the outcome. Such a position brings
into sharp relief the methods by which we seek to ensure ‘equivalence’ of student
learning outcomes. These methods are still heavily influenced by notions of‘identicality’ such as common curricula and centralized examination marking”(Woodhouse and Carroll, 2006, p.85).

	
  
24

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

These opposing views are also expressed by transnational students who have clear
expectations that curriculum should be equivalent, yet contextualized to meet theirneeds. If for example, the content is too Australian-centric, transnational students
have shown to be critical in student feedback (McLean, 2006). The result of this has
been a universalizing of content.
Removing location-specific content is often necessary to avoid
confusing offshore students, but by trying to universalize a course,
lecturers run the risk of abstracting curriculum from real-world contexts,
and thereby elevate the status of 'universal' to many locally and culturallybound ways of thinking, communicating and working. The question weare faced with is why, despite the widespread agreement on thedesirability of adapting and tailoring transnational programs to suitspecific student groups, does it seem to happen so rarely (McBurnie andZiguras, 2007, p.65).

Transnational students also want teaching standards to be equivalent to Australianstandards, yet flexible to meet their needs (Leask, 2006). When the home regulatory
system dominates, an institution is torn between meeting the demands of its
transnational students, providing what is known to be good practice, and ensuring
standards are near to equivalent by delivering exactly the same curriculum in the sameway. The notion of contextualization suggests that standards are moving away fromequivalency and therefore inferior. Navigating between notions of equivalency andcomparability for different types of standards entails risks for the institutions that
could potentially lead to a loss of reputation, loss of commercial return and closure ofa program. For some institutions, the low-risk approach means simply having

	
  
25

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013equivalent standards across as many dimensions as feasibly possible. Whileequivalent standards in transnational education may reduce the potential risk for theawarding institution, it may not necessarily suit the needs of the host institution or its
students.

Shifting interpretation of transnational standards
For the past eight years the Australian Universities Quality Agency (AUQA) has had
the task of auditing transnational education and ensuring compliance with the
National Protocols. The audits provided a public assurance of quality. The fact thattransnational education has the potential for being ‘high-risk’, and that programs
being delivered in another country provide significant signals about the quality ofAustralian education, the government felt that AUQA should scrutinize transnational
activities more closely. In 2003 the Australian government allocated funding to audit
transnational programs, which included visiting partnerships overseas as well as
speaking to staff and students. Since 2003, AUQA has conducted between two andfour transnational audits for every university that has programs offshore.

Greater levels of scrutiny in transnational education had had some effect onuniversities. It is no coincidence that since AUQA began auditing transnational
education in 2003, the number of transnational programs dropped significantly. In
2003, Australian universities reported 1569 transnational programs. In 2007 this had
dropped to 1002 and in 2009 to 889 programs (Universities Australia, 2009). Despite
this, the number of students enrolled in these programs continued to rise between
2003 and 2009. This suggests that there was a consolidation and withdrawal ofprograms with low enrolments. Media reports suggested the withdrawal was largely

	
  
26

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

due to potential reputational risk and the lack of commercial return (Armitage, 2007).
Of the programs that remained, AUQA auditors largely agreed that Australiantransnational education was comparable with their home institutions (Woodhouse andStella, 2008).

While there are considerable differences in opinion about the assurance of quality andthe effectiveness of external auditing (Anderson, 2006), AUQA audits were useful inthat programs and appropriate standards could be contextualized. The audits provided
a forum to consider informed judgments and different interpretations of academic
standards. The diverse social and cultural settings for transnational education make it
important to contextualize standards.

Recent changes in Australia’s regulatory system raises a number of questions of howtransnational standards will be interpreted in the future. Since 2011, AUQA has been
replaced with the Tertiary Education Quality and Standards Agency (TEQSA) and isdeveloping a Higher Education Standards Framework.

The Higher Education

Standards Framework (DEEWR, 2011) has five components:•

Provider Registration Standards•

Information Standards•

Qualification Standards•

Teaching and Learning Standards•

Research Standards

Subsumed within Provider Registration Standards is a sixth element called Provider
Category Standards. This section will also contain a revised set of National Protocols.

	
  
27

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

These will describe the principles that govern each type of higher education
institution and provide a set of minimum standards. It is unclear at this stage whether
the notion of equivalence for offshore campuses and comparability for third-party
partnerships in transnational education will remain. Information standards deal withthe collection and publication of data. A website called ‘myuni’ is planned for launch
in 2012 and will contain a range of information relating to standards. Qualification
standards largely revolve around a revised Australian Qualifications Frameworkdescribing the expected graduate outcomes at different levels of education.
Underneath this may be the development of subject-level standards described aslearning outcomes but this is yet to be confirmed. This would broadly follow the UK
benchmark statements that provide external reference points for setting and assessing
standards in institutions at the subject level. Teaching and learning standards isperhaps the most difficult and contentious area. The setting and assessment ofteaching and learning standards is opaque and complex. It is not clear for example,
whether standards will be set according to institutions’ own missions and goals,
against national or international standards (Thompson-Whiteside, 2011b). Lastly there
are research standards, which are likely to be assessed through the Excellence in
Research Australia (ERA) initiative, which collects research data to assess research
performance within institutions.

While many of these standards are under development it is clear that by withdrawing
from an auditing process TEQSA will be relying much more on quantitative data andperformance indicators. A range of these potential indicators can be seen from Table
1.0 extracted from Coates (2010). The integrity and reliability of this data becomes

	
  
28

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

paramount. As Coates argues, “it is vital that indicators are valid, relevant to key
phenomena, stable across contexts, transparent, non-trivial, responsive to change,
auditable, efficient to collect, preferably readily available, as simple as possible,
quantifiable and generalisable” (p.6).
Table 1.0 Indicators of education quality extracted from (Coates, 2010).

Higher Education
Learners

Higher education
Teachers

Higher education
institutions

Outcomes

Processes

Inputs• Graduation rates• Graduate destinations• Learning outcomes• Graduate capabilities• Work readiness• Satisfaction• Student
engagement• Retention and
Progress• Entry levels• Entry pathways• Student diversity• Student
characteristics• Student aspirations• Teaching experience• Teaching resources• Teaching processes • Staff characteristics• Course• University
management
enculturation• Support systems• Educational
resources• Curriculum• Institutional growth• Institutional
reputation• Community
engagement• Academic
governance• Academic
management• Academic culture• Staff development• Quality systems

	
  
29

• Institutional
characteristics• Institutional
resources• Industry
engagement• Graduate
capabilities• Work readiness• Satisfaction

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

The problem in using a range of these indicators for transnational education is thehighly contextualized nature of teaching and learning. The reliance of quantitative
indicators in transnational education raises potential problems for transnational
education for a number of reasons.
First, the collection of data in transnational education is poor (Garrett and Verbik,
2004, Verbik and Jokivirta, 2005). The fact that students are based offshore from
Australia means that the Australian government relies heavily on individual
institutions collecting the data. In some cases institutions will collect enrolment data
centrally but quite often the collection of data is done in individual departments.
While Australian institutions typically report enrolment data to the government there
is a lack of data concerning teaching and learning. Until now the public assurance ofquality was done through an auditing process and largely focused on institutional
processes. As a result the quality assurance of transnational education has largely
been framed around institutional processes of teaching, assessment and themoderation of assessment. Most of these processes do not necessarily involve thecollection of data. As a result there is little comparative data analysis between
offshore students and onshore students.

Second, one could argue that even if the data were to be collected, it would be invalid
to compare offshore students with onshore students. Comparing data across culturally
and socially diverse settings, across different locations is bound to be complex. Some
indicators are likely to be equivalent but others are likely to be different and thesedifferences can have multiplying affects. The processes of teaching and learning are
dynamic, complex processes and not easily measurable as discrete activities. Even if
some standards were stable or equivalent, it does not necessarily mean that all the

	
  
30

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

other standards would be equivalent. For example, if entry standards and curriculum
were equivalent, it does not necessarily mean that teaching, learning or graduate
standards are equivalent. Comparisons of teaching and learning standards using
purely quantitative data have the potential to be misinterpreted.
Third, the emphasis on quantitative data has the potential to create a situation ofabsolutes. If data between onshore and offshore students are compared and notequivalent then one is presumed to be inferior. There is no contextualization of thedata. Of course, if the policy settings (e.g. the Provider Registration Standards andthe National Protocols) allow for comparable standards then the question is what
difference is acceptable? How does one interpret the differences that inevitably will
occur in the data?

The shift towards a more precise, quantifiable assessment of standards has potential
ramifications for transnational education that has to be fully understood. Where
audits allowed for a contextualization of standards, a standards-based architecture thatis more ‘light-touch’ and data driven has the potential to highlight differences thatexist for very good reasons. If equivalent data between onshore and offshore shows
equivalent standards, then logically, data that shows significant differences suggests
notions of one having inferior standards to the other.

Ensuring equivalent data

between onshore and offshore is likely to be more difficult depending on the mode ofdelivery, the level of autonomy and the amount of contextualization that takes placein the classroom.

By examining the Two Dimensional Typology in Figure 1.0

developed from Davis, Olsen and Böhm (2000), it is likely that a data-driven
standards framework will become more risky for transnational education in thebottom right quadrant.

	
  
31

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Th
er
e
s
u
l
ti
sl
i
k
e
l
yt
od
r
i
v
ei
n
s
t
i
t
u
t
i
o
n
sa
wa
yf
r
o
mc
e
r
t
a
i
ni
n
t
e
r
n
a
t
i
o
n
a
lp
a
r
t
n
e
r
s
h
i
p
s
,
a
n
dc
e
r
t
a
i
nt
y
p
e
so
ft
r
a
n
s
n
a
t
i
o
n
a
ld
e
l
i
v
e
r
ymo
d
e
l
s
.Au
s
t
r
a
l
i
a
ni
n
s
t
i
t
u
t
i
o
n
sa
r
el
i
k
e
l
y
t
owa
n
tg
r
e
a
t
e
rc
o
n
t
r
o
la
n
dc
e
r
t
a
i
n
t
yo
v
e
rt
h
e
i
rt
e
a
c
h
i
n
ga
n
dl
e
a
r
n
i
n
gs
t
a
n
d
a
r
d
s
.
Wh
e
r
et
r
a
n
s
n
a
t
i
o
n
a
lp
r
o
g
r
a
msh
a
v
eh
i
g
hl
e
v
e
l
so
fi
n
v
o
l
v
e
me
n
tf
r
o
mt
h
i
r
dp
a
r
t
y
p
r
o
v
i
d
e
r
s
,i
nt
h
ef
o
r
mo
ft
e
a
c
h
i
n
g
,t
h
ec
o
n
t
e
x
t
u
a
l
i
s
a
t
i
o
no
fc
u
r
r
i
c
u
l
u
m,a
n
d
/
o
r
a
s
s
e
s
s
me
n
t
,t
h
er
i
s
k
so
fd
e
mo
n
s
t
r
a
t
i
n
ge
q
u
i
v
a
l
e
n
c
yi
n ad
a
t
a
d
r
i
v
e
ns
t
a
n
d
a
r
d
s
f
r
a
me
wo
r
k
,
a
r
el
i
k
e
l
yt
ob
eg
r
e
a
t
e
r
.

32

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Conclusion
The recent shift in Australia away from quality assurance and auditing of institutions,to a more precise standards-based framework has considerable implications forAustralian transnational education. A standards framework that relies heavily on thecomparison of data has implications to drive institutional behaviour away from certain
forms of international collaborations and types of transnational delivery.

The

comparison of data does not sufficiently allow for interpretations and acontextualisation of complex teaching and learning processes in different cultural
settings. When policies require equivalent standards in transnational education, thenthe risks for transnational may be too high. Even if policy settings allow forcomparable standards, any differences in data will be considered a risk to standards.
The notion of difference and the desire to reach equivalency fails to recognize thecomplexities of transnational education and ultimately is unproductive in generating
mutually beneficial, long-term, sustainable partnerships. To minimise any potential
differences, Australian institutions are likely to constrain the types of international
partnerships, the types of transnational delivery and reduce the number of programs.
This in turn will have implications for countries that use transnational education as away of capacity building. It is likely to restrict access to Australian higher education
for students in those countries.

	
  
33

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

References
Adams, S. (2001). Transnational Education Project Report and Recommendations.
Confederation of European Union Rectors’ Conferences.
AEI. ( 2005). Transnational Quality Strategy. In: AUSTRALIAN EDUCATIONINTERNATIONAL (ed.). Canberra: Australian Education International.
AEI. ( 2008). Good Practice in Offshore Delivery: a guide for Australian providers.
In: AUSTRALIAN EDUCATION INTERNATIONAL (ed.). Canberra.
AEI. ( 2010). Transnational Education in the Higher Education sector. In:
AUSTRALIAN EDUCATION INTERNATIONAL (ed.). Melbourne.
Anderson, D.( 2006). Assuring Quality/Resisting Quality Assurance: Academics'
responses to 'quality' in some Australian universities. Quality in Higher
Education, 12, 161.
Armitage, C. (2007). Cost, rivals force offshore reversal. The Australian, June 27.
AVCC. ( 2005a). Australian University Offshore Quality Assurance: Refining not Redefining. Canberra: Australian Vice Chancellors Committee.
AVCC. ( 2005b). Code of Practice and Guidelines for Australian universities:
Provision of Education to International Students. Canberra: Australian Vice
Chancellors Committee.
Bohm, A., Davis, D., Meares, D. & Pearce, D. (2002). Global student Mobility 2025:
Forecasts of the Global Demand for International Higher Education. Sydney:
IDP.
Bradley, D., Noonan, P., Nugent, H. & Scales, B. (2008). Review of AustralianHigher Education. Canberra: Department of Education Employment andWorkplace Relations.

	
  
34

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Coates, H. (2010). Defining and monitoring academic standards in Australian higher
education. Higher Education Management and Policy, 22.
Connolly, S. & Garton, J. (2007). A Strategic Approach to QA for Transnational
Education Programs. AIEC. Melbourne.
Davis, D., Olsen, A. & Bohm, A. (2000). Transnational Education providers, partners
and policy: challenges for Australian institutions offshore. In: Davis, D.
Olsen, A. Bohm, A. (Eds.). Brisbane, Australia: IDP Education.
Deewer.( 2011). Regulatory and Quality Arrangements Diagram [Online]. Canberra:
Department of Education Employment and Workplace Relations. Retrieved
January 12, 2011 from:http://www.deewr.gov.au/HigherEducation/Policy/teqsa/Documents/HIEDArr
angements_Diagram.pdf.
DEST. (2005). A National Quality Strategy for Australian Transnational Education
and Training: a discussion paper. In: DEPARTMENT OF EDUCATIONSCIENCE AND TRAINING (ed.). Canberra.
DETYA. (2002). National Protocols for Higher Education Approval Processes. In:
DEPARTMENT OF EDUCATION TRAINING AND YOUTH AFFAIRS(ed.). Canberra.
Garrett, R. & Verbik, L. (2004). Transnational Delivery by UK Higher Education,
Part 1: data & missing data. Observatory on Borderless Higher Education, 18.
Harvey, L. (2004). War of the Worlds: who wins in the battle for quality supremacy?
Quality in Higher Education, 10.
Heffernan, T. (2005). In search of "the vibe": Creating effective international
education partnerships. Higher Education, 50, 223-245.

	
  
35

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

IEAA. (2006). Outcomes of Universities Transnational Education good practiceprojects Canberra: International Education Association of Australia.
Ilieva, J. & Goh, J. (2010). Measuring the Internationalisation of Countries’ Higher
Education Systems: A comparative perspective from 11 countries. AustralianInternational Education Conference, October 14 2010 Sydney.
Knight, J. (2005). Borderless, Offshore, Transnational and Cross-border Education:
definition and dilemmas. The Observatory on Borderless Higher Education.
Lane, B. (2011). Agency makes surprise move to axe audits The Australian, 23
November.
Leask, B. (2006). Keeping the promise to transnational students - developing idealteacher for the transnational classroom. AIEC 2006. Perth.
Leask, B. (2007). Internationalisation of the curriculum and intercultural engagement- a variety of perspectives and possibilities. AIEC. Melbourne.
McBurnie, G. & Ziguras, C. (2007). Transnational Education: Issues and trends inoffshore higher education, New York, Routledge.
McLean, V. (2006). Auditing Transnational Partnership Programs. In: BAIRD, J. H.
(ed.) Quality Audit and Assurance for Transnational Higher Education.
Melbourne: AUQA.
Sanderson, G., Yeo, S., Thuraisingham, T., Briguglio, C., Mahmud, S., Singh, P. H. &Wallace, M. (2010). Interpretations of Comparability and Equivalence around
Assessment: Views of Academic Staff in Transnational Education. Australian
Universities Quality Forum: Quality in Uncertain Times. Gold Coast.
Thompson-Whiteside, S. (2011a). Moderation: a dialogue for assuraing andcontextualising transnational standards. International Conference of
Education, Research and Innovation. Valencia, Spain.

	
  
36

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Thompson-Whiteside, S. (2011b). Who sets the standards in higher education?
Australian Association of Institutional Research Forum. Gold Coast,
Australia.
UNESCO. (2001). Code of Good Practice in the Provision of Transnational
Education. Lisbon: UNESCO.
UNESCO. (2005). Guidelines on Quality Provision in Cross-border Higher
Education. United Nations Educational, Scientific and Cultural Organisation.
Universities Australia. (2009). Growth of offshore programs 1996-2009. Canberra:
Universities Australia,.
Van Der Wende, M. (1999). Quality Assurance of Internationalisation andInternationalisation of quality assurance. Quality and Internationalisation inHigher Education. Paris: OECD.
Verbik, L. & Jokivirta, L.(2005). National Regulatory Frameworks for Transnational
Higher Education: Models and Trends, Part 1. London: Observatory on
Borderless Higher Education.
Woodhouse, D. & Carroll, M. (2006). Quality Assurance Issues in Transnational
Higher Education – Developing Theory by Reflecting on Thematic Findings
from AUQA Audits. In: BAIRD, J. H. (ed.) Quality Audit and Assurance forTransnational Higher Education. Melbourne: AUQA.
Woodhouse, D. & Stella, A. (2008). Borderless Higher Education ‘Down Under’:Quality Assurance of Australian Cross-Border Initiatives. London: The
Observatory on Borderless Higher Education.

	
  
37

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  

	
  
38

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

A Methodology to Develop Ontologies for Emerging Domains
Chai Meenorngwar

Author’s Note
The author would like to thank the Royal Thai Government and Valaya Alongkorn
Rajabhat University.

39

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Abstract
The characteristic of complex, dynamic domains, such as an emerging domain, is thatthe information necessary to describe them is not fully established. Standards are notyet established for these domains, and hence they are difficult to describe and present,
and methods are needed that will reflect the changes that will occur as the domains
develop and mature.

This research proposes the Liverpool Metadata or LiMe

methodology to develop an ontology and organise the knowledge that is necessary fordeveloping the domain environment descriptions. Its aim is to capture Knowledge
Information (KI) from research articles and translate this into semantic information
with web description languages such as XML(s), RDF(s), and OWL. LiMe represents
an Ontological Framework, which provides the concept characteristics, represented asa concept framework that specifies conceptualisations of the knowledge. LiMe
supports the Semantic Web development. “e-Learning” has been chosen as an
example of an emerging domain in this research. The characteristics of e-Learning
concepts will be extracted from research articles of journal websites such
ScienceDirect, Springer, etc and represented as knowledge. LiMe also explicitly
represents how these concepts are developed and evolve to represent the domain.
Keywords: E-learning domain; emerging domain; knowledge information; ontology;
semantic Web

40

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Introduction
Overview
The range and quantity of information available via the Internet today has createdwell-known problems of information overload, including difficulty of access andproblems of selecting information that is appropriate and reliable. To address these
problems, ways were required to categorise and organise information for access byusers. The idea of using multiple sources can facilitate the reliability of knowledge,
but increases the need for effective knowledge management.

A domain of knowledge can typically be seen from different perspectives. Also,
information about them is diverse and possibly contradictory. Think for example ofthe huge mass of information contributed every day on the Internet. Therefore,
methods are needed to classify and identify information to find reliable sources toconstruct the knowledge.

In addition, information can change and be flexible, based on time and need. For
example, complex domains such as software development have a lot of platforms andstandards. Knowledge or concepts in the domain have been defined or represented indifferent ways. Therefore, users find it difficult to choose the suitable system orconcepts for their own environmental needs.

This shapes a complex and unstructured environment where unstable concepts andinformation are contributed all the time in a domain. The representation itself of thedomain is also difficult. It needs methods to capture new concepts, organise existing
concepts, and translate into well-formed information that could be shared and reused.

41

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Objectives
The work in this research sits broadly in the field of Knowledge Management (KM).
KM (Eriksson, H., 2004) is identified as the capabilities and communication that
include: (1) converting individual to group-available knowledge; (2) converting data
to knowledge; (3) converting text to knowledge; (4) connecting people to knowledge;(5) connecting knowledge to knowledge; (6) connecting people to people; and (7)connecting knowledge to people. It is represented as the combination of documents
and ontology that can help organisations describe, store, catalogue, and retrieve
information in a systematic manner.

This research introduces an approach that can help the users to classify theirinformation and to represent it with a well-formed structure. The approach provides
an ontological framework to structure one individual existing domain. This work
focuses in particular on the problem of information management in an organisation.
Information within an organisation needs to be accessed for different purposes.
Experiences from individuals in the organisation help forming the common
understanding, which could used or reused to develop new information, therefore it
needs to be made shareable and reusable. In fact, individual experiences are a very
important source of knowledge. For examples, researchers use the educational
experiences to find the information about their experiences, governors used theworking experiences to organise their daily information, and teachers collected theinformation from books, experiments, and so on to prepare their courses.

42

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

The aim of the research described in this research is to investigate issues involved in
the representation and management of knowledge arising in an emerging domain. A
number of techniques have been used for representing domain knowledge. In mostcases, these methods assume the existence of a well-defined body of knowledge thatcan be assumed to be reliable and definitive, and needs only to be organised
appropriately. In the case of emerging domains of knowledge, however, theseassumptions are not valid. In this case, the “body of knowledge” is incomplete andconstantly changing, and may include significant errors, inconsistencies, andinstances of different assumptions, conclusions and terminology. Only when thedomain reaches a state of relative maturity can these issues be resolved definitively.
Meanwhile, however, there remains a need for researchers and practitioners to makeuse of the knowledge while it is in this state of evolution.

This research proposes a framework, Liverpool Metadata (LiMe), as the way totransform the individual experiences into relevant information for a particular domain
by applying an ontology approach, structuring these experiences in terms of concepts
and the relation between concepts. Concepts are defined from different perspectives
under the same domain. These could be redefined, reused and described as
specification of the particular domain. The development processes of the LiMe
methodology is described in the following section. LiMe provides techniques tomeasure the relation between the concepts in the ontological framework. This allows
to store and access with the other. The relation between the concepts presents asknowledge to improve the framework from new information. It is described with thewell-defined descriptions such the formal language such XML(s) and RDF(s).

43The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

An ontology is a shared description of concepts and relationships in domain
knowledge. It consists of terms, their definitions, axioms relating to them, and ataxonomy organizing them. The main objective of an ontology is to enablecommunication and knowledge sharing by capturing a shared understanding of terms
that can be used by humans and programs. It has been argued for the use ofknowledge representation techniques capable of reflecting the situated nature ofhuman cognition (Gahegan, W.P.a.M., 2007). It also facilitates the sharing and reuseof information and can reduce the analysis, design, and development time of complex
systems.

Within the body of knowledge to represent, a distinction can be made between
information coming from referential sources and information coming from practicalsources.

Referential sources use documents such as a research paper which provides reliability
to the domain. Practical sources use the working experiences such tasks, activities,
etc. In case study section, the approach will be exemplified by means of two casestudies, one in the educational field (e-learning case study) and one in the governancefield (e-inspection case study).

44

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Domain	  Level	  Requirement	
  I	
  Requirement	
  II	
  

Requirement	
  III	
  Ontology	
  Level	  Figure 1: This description of LiMe approach.

For both the above cases, information was developed with the cyclical processes.
Firstly, new concepts were defined from individual experiences and formed thestructure of knowledge. This was represented as a tree of concepts. Secondly, eachconcept was linked with the other concepts forming relationships. Users are helped
define and arrange these concepts by the LiMe environment. LiMe introduces
similarity of concepts in the ontological framework and provides the user withdirections for descriptions: generalisation and specification. Therefore, the users candefine the appropriate descriptions for each individual information environment.

Figure 1 illustrates the main spirit of LiMe approach: users characterise their own
requirements on the domain and are helped to represent them in an ontology.
Organisations typically have to deal with lots of information which is unstructured
and difficult to reuse and share.

45

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Case Studies Outline
In this study, two different organisations, educational and government environment
will be used to show how an ontological approach can help classify information incomplex scenarios.
The e-Learning case study demonstrates the use of referential sources to capture
online learning concepts from research papers to shape the domain knowledge for the
Valaya Alongkon Rajabhat University. E-learning is a good example of an “emerging
domain”, that is a domain which has the particular additional difficulty that thecurrent body of knowledge is not stable (O’Hara, C.B.a.K., 2007). Research into eLearning is currently very active (M del Puerto P., 2008), and the concepts involved
are constantly also being redefined and introduced in different ways. In the casestudy described in this research, human researchers deal with an emerging domain bya process of continuous review of published literature, from which the current
consensus emerges. In the same way, published research papers will be used as the
input resources of this research.

The second case study will demonstrate the use of practical sources to represent the
information that is used for describing the problems in the project inspections task for
the Royal Thai Government. It will later describe these case studies in detail.

46

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Research problem
In the Internet era, people are using the information from the websites or place thatthey connect to. Technology provides a convenient living style. However, there aresome problems for information developers in case of complex and ever changing,
emerging domains, such as in the government sector or e-learning. The increasing
amount of information, especially internal information such documents, projects,
tasks, requests, etc, contributes to the unstructured nature of information.

The obstacle of accessing the appropriated information needs much more time andhigh development costs. This research will provide processes of information
classification by using an ontology approach. This is the beginning of this research
problem. The research question and problem is described below:l

Research question: Is it possible to organise the information of an emerging
domain by using an ontology approach?

lHypothesis I: The experiences or information from the different people could be
represented with the ontology. These come from the individual person of theorganisation.

l

Hypothesis II: Information from an emerging domain could be used and shared
the information by using the existing ontological framework.

l

Hypothesis III: Semantic Web could be developed from the existing ontologicalframework as input.

In this research, an attempt will be made to define ontologies to facilitate environment
description and represent a complex, frequently changing domain. A tool, LiMe has
been implemented to capture the relevant information from a particular domain. The

47

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

objective is to transfer information and data from paper or oral communication to arepresentation of the knowledge in a computer system.

Literature review
In this research, an ontology development approach is proposed for capturing
information and knowledge in complex domains, such as an emerging domain or adomain involving flexible information, various approaches and methods that change
constantly. E-Learning systems will be used to illustrate a domain of the former kind,
while a government setting will be used to illustrate a domain of the latter. In theimplementation of the research, languages such as XML(s), RDF(s), and OWL areused to describe the domain environment. This chapter reviews the literatures tosupport the research approach. The section has the following four main sections:l

Knowledge: problems such as using knowledge in various platforms, describing
knowledge with different approaches, time to develop knowledge in theorganisation, etc. Knowledge development is introduced to facilitate and solvethese kinds of problem.

l

Ontology: ontology technology could be used to organise the knowledge.

lAn e-Learning system: the e-Learning systems has introduced as domain
example.

l

Semantic Web: it has been used to represent the flexible knowledge informationin the domain.

48

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Knowledge and Information
Knowledge characteristic has been classified as degree of articulation and aggregation(Cooper, 2007). It is information in the context of other information, such as therelationship between data, information, knowledge and wisdom represented in it.

Knowledge is different from information when it has been used or introduced asproblem solutions. The knowledge definitions are concerning on the goal of the
problem. For example, the knowledge (in term of learning/teaching of online
environment) is the information about the courses in the pedagogical curriculum.
Knowledge is the information which solved the particular problem.

Information is derived from raw data in the events. For example, the registration datasuch as student information, courses registration details, are contributed when thestudents choose the online courses. Information could be constructed from these datasuch as registration table, numbers of the courses that open for selecting,
instructor/teachers/allocating to the courses.

Knowledge has been defined as classification, without the classification human could
be thought, action, or organisation such example of Dewey Decimal Classification(DDC) which is a method that uses in US Library of Congress classification(Wingyan, 2007). Knowledge is unstructured information provided by different
sources such research papers or working experiences. The next section will present
some techniques to manage knowledge.

49

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Ontology
This research is concerned with building an ontological e-Learning requirements
framework to facilitate the users or the developer to understand and use it forreferencing, describing, searching, retrieving their own environment from theacademic research methods or article as knowledge resources. An ontology specifies a
common conceptualisation, independent of data model, and this may be presented as
Semantic Web. It extracts data user contributions, and captures data as people share
their knowledge in terms of classes and relations between classes. It represents
existing things by illustrating and structuring the knowledge from important
vocabularies. Basically, people adopt their vocabularies to the ontologies. Then,
description languages such XML(s), RDF(s), OWL have been introduced to encode
the structured data and tie it with common vocabularies as classes, properties, andrelations with well-maintenance namespaces.

The domain will be represented as a common framework and helps to integrate or
exchange data from multiple resources. The consistent knowledge of a specificdomain environment is captured and combined with different information sources.
Then, a reasoning approach is needed to support to interpret this framework as
semantic knowledge.

In ontology, the characteristics of an interest domain have been described as concepts
or entities, properties of the concepts, and relations between concepts that include theconstraints (Patil, 2005). Thus, it will be used as value-mapping (support the various
format or data), and scalability (depending on the context of data) (N.Huhns,
K.M.a.M., 1997).

50

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

An ontology is a specification of a concept or property as knowledge (Sheng, 2004) ora concept framework (Zhang, 2006) and content management that consists of fiveprimitives (Wang, 2006): class, relation, function, axiom, and instance. It specifies aconceptualisation of a domain in a term of concepts, attributes, relations, instances,and theories. A concept is a set of individuals or objects in a domain. An attribute isused to depict an intrinsic feature of objects. In addition, the domain scopes orobjectives of the domain will be described with concepts and relations. Semantictranslation determines the similarity between terms as instances of different domains
and maps instances from one to other.

In practical terms, Semantic Web technology uses Ontology abilities to communicate
between human and computer by providing an explicit specification for theconceptualisation of the existing domain. The classic Web will be extended with themeaning of concepts on Semantic Web which could also be shared and reused.

Next, it will explain examples of the research areas that used ontologies to describe
their domain environment.

In Information Retrieval systems (Hwang, M.K., 2007), ontology is used to create,query, inference, and management information that help users to edit, delete, andmodify the existing knowledge in the domain. In order to retrieve the informationfrom the ontology, the reasoning and processing will be used in the query engine.

For example in the tourism domain (Dai, B.A.W., 2005), it is not only informationsuch as the accommodation profiles (details, facilities, etc) that is annotated with RDF

51

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

metadata which could be retrieved but also

tourism information such as water

quality, places, etc could be annotated as semantic data and used for intelligent
search (Sebastian Hübner, R.S., 2004).

Wingyan (2007) proposed Web directories to use ontologies to organise voluminous
information into hierarchical structures, and help users to quickly locate relevant
information and to support decision-making.

E-Learning
An e-Learning system is an education system that is provided in an online
environment, usually via the Internet. Various related terms include virtual classroom,
online learning, web-based learning, computer based learning, web instructions, etc.

The use of the Internet in education has the potential to motivate students andteachers, increase student participation and interaction in the classroom, and provide
students with a more active role in their learning and increased autonomy in the
educational process. While teachers are requested to use the capability of the newhigh technology to facilitate learning processes, students are encouraged to improvetheir learning through computer and networked-based activities.

For example, the Ubiquitous e-Learning (Norm Friesen, R.M., 2005) is a formal
education which not only outside the classroom but also outside the education
environment such as workplace, street, home.

52

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

In addition, an e-learning environment (Norm Friesen, 2005) regards teaching as acontinuous process transferring knowledge with delivery in different forms such as
offline and online learning; self paced and live learning; structural and unstructured
learning; formal and informal learning.

With LiMe, a learning environment to be developed as e-Learning will be designed
and organised, and the environment based on individual requirements. These
requirements will be transformed as a common understanding framework whichavailable to be modified by each user. The different facilities such human or
knowledge experiences, technologies, learning materials, etc could be solved by using
this common understanding framework.

Semantic Web
Semantic Web has been used to produce a semantic context-aware knowledge
management framework that enables to integrate knowledge discovery, retrieval, andreuse (Norm Friesen, 2003).
Semantic Web technologies use smart tools to assist the system administrators tomanage and control various kinds of problem. The requirements of the domain
environments could be represented without misunderstanding by extracting andmodelling the knowledge from the various documents and using Ontology to accessand manage knowledge. Consequently, the common understanding of concepts ispresented as semantic knowledge.

One of the most important aspects of the Semantic Web is searching knowledge fromontologies. Rules of representation have been designed in machine understandable

53

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

form (Nenad, 2002) facilitates to achieve the semantic information. However, it needs
mechanisms and background knowledge about the domain for processing on
ontologies such as updating or adopting their knowledge and reasoning strategies.

Liverpool MetadataThis chapter will illustrate the methodology to build and share ontologies forrepresenting an Emerging Domain such as the e-Learning requirements domain, andwill introduce ‘LiMe’ (Liverpool Metadata), as a means to facilitate the description of
the Knowledge Information.

The idea of LiMe is to provide the descriptors or concepts which represented
knowledge that obtained from research papers. In an emerging domain, the research
papers provide the only effective knowledge resources, and using an ontology enables
to describe this as knowledge information from them.

LiMe presents the knowledge specification of domain environment and provides theability to share and reuse knowledge, providing a common understanding among
different perspectives. People often give different names or definitions for the samething, or different things can be described with the same definitions. An ontology
aims to help this kind of problem.

54

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Aim
The characteristic of an Emerging Domain (ED) is that the information necessary todescribe it is not fully established, and hence it is difficult to describe and present, andneeds methods that will reflect the changes which will occur as the domain develops
and matures.

Knowledge/Information Ontologies provide the knowledge or descriptions that arenecessary for developing the domain environment descriptions. The aim is to captureknowledge information from the research papers and convert to Web Description
Language such as XML(s), RDF(s), and OWL.

It represents Knowledge/Information as Ontological framework, in which Concept
Ontologies provide the concept characteristics which are represented as a concept
framework that specifies conceptualisations of the knowledge. Representation
Ontologies use the Semantic Web to illustrate the domain environment based on an
ontological framework.

Characteristics
Methodologies used to develop an ontology have five different techniques: frames
and first order logic, description logic, software engineering, and databases (GomezPerez, 2004).

LiMe uses the database technique and presents the domain with hierarchy of concepts
as tree in the figure 2. It has been designed to store knowledge from information orparagraphs of the research articles. Both information and paragraphs are called

55

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Knowledge/Information (KI) which is a consensual knowledge used to extract theconcepts and their properties as Object Oriented modelling.

Environment will be organised and represent the characteristic of the domain. The
particular environment is the subsystem or sub-organisation that represents thefunctions in the domain. For example, in e-Learning domain, it consists of learning,teaching, and management.

Knowledge/Information in the particular environment is used as referencing resourcesthat defined concept, properties, and instances. This information also facilitates todefine the relation between concepts. Relation is the relationship between two or moreconcepts. LiMe classifies the relation in two different relation categories:
specification and generalisation. It also presents the semantic meaning direction.
Specification is the top-down approach and generalisation is the bottom-up approach.
Both approaches are used to develop trees or taxonomies that are called ontologies inthe domain. A circumstance of domain uses ontologies to exchange the common
understanding and give as a structure framework.
LiMe methodology organises domains as a five-level taxonomy. For example:l

Domain: e-Learningl

Environment: Learning, Teaching, Administration, Infrastructure, etc.

lKnowledge/Information: definitions or meaning, functions, Examples, etc.

lMetadata: Learning Material, Student, Teacher, Learner, Instructor, etc.

l

Properties: Learning process, tasks, etc.

56

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

LiMe Resources
LiMe illustrates a hierarchy of research papers as Web resources and Knowledge fordeveloping the Ontology of a particular domain environment. A research paper isorganised with two parts: Reference Resource and Knowledge. It is introduced as
Web Resources which contain reference information and knowledge. Knowledge will
be classified as Information that is captured from the research paper or the individual
experience which is contributed by the developers.

A research paper contributes information such as research problems, researchmethods, objectives, research results, and conclusion, represented using text, tables,or diagrams. This information is used as Knowledge /Information (KI) for developing
an ontology. LiMe captures KI from the research papers using the individual
experience and background knowledge of the (human) reader. Moreover, LiMe uses
KI to extract or define the concepts that are related to the domain. A concept may be
a general concept or class, a specific concept or property/instance, or a relation
concept that represent the relationship between concepts, instances, or properties.

Instead of searching the knowledge based on keywords from the journal, LiMe
organises knowledge that facilitates to reduce the retrieval time. The unnecessary
article will not be listed. However, the appropriate concepts that facilitate to identify
or describe the knowledge are important, costly, and time consuming process.

LiMe’s Development Cycle
To capture the Emerging Domain (ED), flexible or new concepts are extracted fromthe research domain. LiMe presents these concepts knowledge as Metadata and uses

57

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

to develop ontology. LiMe proposes the development cycle (Figure 2) with four basic
methods: KI identification, Concepts extraction, Ontology development, andRequirements representation.
Knowledge	  Information	
  
KI	
  
Identification	  Metadata	  Extraction	  Semantic	
  Knowledge	  Patterns	  	
  	
  Metadata	  Ontology	  Development	
  Requirements	
  representation	  Ontological	
  Framework	
  Figure 2: This figure illustrates LiMe development cycle.
The information relating to the ED is gathered from research papers. Our aim focuseson transforming the Domain specification to Semantic Knowledge.
Domain specification > Semantic Knowledge

In LiMe, domain specifications will be represented as the requirements from various
researchers that contributed KI included both approaches and results in this researchdomain area. This knowledge will be organised with concepts that extracted from thisknowledge information. Concepts also represent the patterns of knowledge which isused to classify the knowledge categories such as meanings, definitions,specifications, functions, tasks, etc.

An ontology in our research is the knowledge classification. It describes the domain
specification. It translates the KI in each particular environment to ontologicalframework. This framework is the place for interchanging the knowledge in theenvironment and will be interpreted as semantic knowledge with Semantic Web.

58

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Knowledge identification
In order to extract the knowledge from the research paper, LiMe imports theKnowledge/Information by using the academic journal search engine which the
keywords to gather the domain specification from search engines of the academic
journal websites such as Springer, ScienceDirect, IEEE Xplorer, etc. However, thisphase does not an automated mechanism, human still have to choose and find therelated papers.

This process could take a lot of time especially for non-expert

knowledge domain developer within huge related domain articles are listing.

LiMe describes KI as the crucial information or context information that help theusers (developers, researchers, etc) to understand about the domain where could
locate on paragraphs of paper articles such abstracts terms, definitions, notation,abbreviations, examples, approach, results, experiences, discussion, related topics,
and so on.

In addition, LiMe also introduces the patterns of knowledge such as
meaningful/definitions (descriptions), components (properties, instances), restrictions(relations, condition, constraints), etc. which could be added and improved. LiMe willstore these patterns as KI categories and use them to reduce the time of capturing in
the future. In order to understand, the tasks of Knowledge identification have been
represented as follows:1.

Define the scope or particular environment of domain of interest, which is theobjective for developing an ontology. For example, this research concerns on
describing the e-Learning requirements domain, therefore, the objective is to59

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

develop an ontology to annotate requirement in e-Learning domain, to help eLearning researchers or organisation developers.
2.

Define the keywords that related to the domain or scope, such as topics, title,
instances, etc. Instead of only generate keywords from background experiences
consideration, keywords could be found in the LiMe’s thesaurus within theexisting environment framework.

3.

Use the keywords to find the related articles from the academic journal websites.
With the large number of online papers, existing keyword-based searches
retrieve many irrelevant papers that may use a certain word in different contexts;
they might also miss papers when different words about the desired content are
used.

4.

Find the crucial information related to the domain, based on the previous patterns
or categories. A pattern is a kind of context that identifies the relation to thescope, environment, or domain which is not easy to identify. Especially,
different researchers express their knowledge in different ways. Backgroundexperiences of the domain will help to identify the knowledge context from thegeneral information.

5.

Capture the KI from articles and store it to the LiMe system. This KI will alsotranslate to the formal language XML(s). LiMe also captures the article profiles
such as title, author(s), journal, volume, issue, page, and URL. This information
is a reference resource to refer during developing an ontology. Note that LiMe
does not upload the file resource.

6.

Update and improve the pattern identification. All the tasks are repetitive tasks.
LiMe enables the developer to define the patterns which are the contextual
criteria of KI.

60

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Metadata extraction
LiMe produces sets of Metadata of the domain environment which are extracted from
KI obtained from the research domain articles. LiMe presents Metadata in the term of“concept”. In order to extract the concept from the KI, the follow steps are followed:
1.

Find the general or specific topic such as subject or object in the statement. LiMeis concerned with capturing the definitions, components, or functions from theKI. There are various ways to find the concept in the paragraphs: find thespecific concept, find the general or abstract concept, and use experiences todefine the concept.

2.

Given the type of concept, LiMe has four different concept types: class,properties, relations, and instances. Class is the entity or the existing things ofthe environment in the domain. Properties are the specification details of theconcept. Relations are the relationship between concepts, which are propertiesof a concept. Instances are the example objects for concepts. Some concepts
could be both class and properties. LiMe presents concepts as the Object
Oriented model in Class, attributes and objects.

3.

Define an explicit relation hierarchy between the concepts in the same KI. In
addition, properties, and instances are used to specify characteristic of concept.
LiMe uses taxonomies to organise concepts, properties, relations, and instancesin the ontology. LiMe has relation based on type of the concepts.

4.

Compare this topic with LiMe’s thesauri that provide semantic between concepts
such as synonym relationships. Then, update the new concept to the thesauri. A
concept might take different assumptions from different perspectives and be usedin different areas. In order to clarify the definition, LiMe proposes the existing61

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

concepts with an ontological framework that could be specified the definition for
creating the new concept in the thesauri.
5.

Generate tree or taxonomy of the concepts to represent knowledge and alsotranslate this taxonomy to formal description language such as XML(s), andRDF(s). Therefore knowledge is represented with one or more taxonomies froma particular KI as independent descriptions.

At this step, LiMe produces Metadata that will be used to describe the knowledge
from KI. LiMe has classified Metadata based on the three basis functional types
described from the statements in KI. Descriptive Metadata is a concept that describes
the information such as meaning, definitions, etc of the knowledge. Structural
Metadata is a concept that classifies or structures information of the knowledge.
Finally, Administrative Metadata is a concept that describes information such as
constraints, conditions, rules, etc of the knowledge. A set of concepts extracted fromKI will be represented in this task.

Ontology development
LiMe proposes to develop an ontology for representing the Emerging domain. The
development process is mainly integrating the taxonomies constructed from KI of
research articles. An Ontological Framework (OF) is the result of this method. It
enables developers to communicate and interact to the Emerging Domain bycontributing the common understanding of concepts. This is a structure informationthat objective, accessible reusability, and flexible accomplishment.

62

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

In order to develop an ontology, LiMe proposes two basic processes: Similarity
measurement, and Taxonomies integration as following.

Similarity measurement
LiMe uses the similarity between two concepts to reduce the redundancy and presents
consistent concepts. Similarity could be easily detected by humans, whereas
computers need to evaluate parameters to identify the similarity.

Currently, there are some methods that contribute to similarity algorithms such as
Information-based similarity (Al-Mubaid, 2006), functional and textual based method(Ganjisaffar, 2006), and similarity graph (Andreasen, 2003). And, the relationships
between concepts could be described in the terms of Synonymies, Hyponymies, andOverlapping (Maria Ruiz-Casado, 2005). Synonymies denote that two or moreconcepts have the same meaning. Hyponymies denote that a concept has more than
one meaning. Overlapping indicate that concepts are neither synonymies nor one
hyponymy of each other, but represent to some extent the same reality.
LiMe proposes to use the combinations of two techniques to compute the similarity
between concepts: first, using the weight of the concepts, and secondly, using the
distance between concepts.

Weight-based technique. This similarity method described in (Ganjisaffar, 2006) isbased on functional and textual information. The concept similarity function
calculates, from a pair of concepts, a real number between 0 and 1, expressing thedegree of similarity between two concepts, based on two characteristics: Taxonomy

63

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

based concepts and weight Information Content. The “1” value indicates that a pair ofconcepts are strongly similar whereas “0” indicates that they are different.

Edge-based technique. This similarity technique counts the edges between concept
c1, and c2. For example, (Zhumin, 2006) describes Wu and Palmer algorithm that
calculated the similarity between concepts as following.
Sim(c1,c2) = 2 x / ( x + 2 x )Where and are the length of the path from c1 and c2 to their most specific common
super-concept c3, and is the length of the path from c3 to the root of the hierarchy.

Taxonomy integration. In order to integrate taxonomies from a domain, LiMe focus
on a similarity measurement. The relation between common concepts will be definedwith “is-A” and “part-Of” relationships. The “is-A” relation is used to express that apair of concepts have fully similar characteristics. The “part-Of” relation is used toexpress that there are partly similar characteristics between two concepts.

LiMe uses both a Top-down and a Bottom-up approach to integrate and express theknowledge direction in taxonomies. Top-down approach is used to annotate a more
abstract concept with the specific existing concepts. This could be extracted not only
from the research papers but also provided by domain experts as their backgroundexperiences.

To optimise taxonomies, LiMe uses Term matching technique patterns (Asanee,2004) that integrate the similarities concepts to structure, form, or extend thetaxonomies with four different cases.

64

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

A term matching technique is used to integrate a taxonomy which has a concept that
could be expressed with relations to a different taxonomy. In additional, a consistency
concept is the similarity between concepts in different taxonomies and could be
expressed with the”is-a” or “part-of” relation. LiMe calls an existing taxonomy that is
extended with a consistency concept as a core hierarchy.

Requirements representation. LiMe is concerned with representing Knowledge thatis constructed from individual user perspectives. LiMe defines knowledge provided
from the individual research article as requirements.

In order to provide a dynamic or flexible representation, a distinction can be made of
the source contributing knowledge in two basic types: the reference requirements andthe user-defined requirements. This will allow the users or developers to have aflexible opportunity to define their knowledge as knowledge template.
Research	
  Knowledge	
  

User	
  
KnowleReference	  requirements	  User-­‐Defined	
  requirements	  Knowledge	
  Blog	  Interactions	  Ontological	
  Framework	
  Composed	  by	
  Description	
  Languages	
  Taxonomies	
  Figure 3: Knowledge Blog in LiMe.

65

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

The reference requirements are perspectives on the knowledge, the information fromthe research paper or articles. The user-defined requirements are contributed by theusers or developer in the organisation and will be used to understand the backgroundknowledge about the domain. It is possible that they do not have the knowledge orunderstanding about the domain environment.

With the various different knowledge perceptions, a flexible representation approachis required to handle the various information formats. This work proposes a semantic
blog, the Knowledge Blog (KBlog), to organise and describe the different
understandings of the Ontological Framework in the domain (Figure 3).

The main contribution of the Knowledge Blog is the idea of using a Blog to present
the conceptual knowledge. With Blog technology, the knowledge contents aregathered from the individual requirements and research contributions on the web. The
KBlog provides the interface to the knowledge of the domain as mechanism ofknowledge annotation and facilitates the users to look and find across the blog
comparing their knowledge with the others.

66

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Case Studies
In order to illustrate how to use the ontological Framework, this section will present
the results from two case studies: an educational and a government based case. LiMehelped produce the ontological framework which was used to develop Semantic Web
solutions in each domain.

The following evaluation has been carried out, and feedback was obtained on thedevelopment. These web based applications are fully implemented and have beenused in a real environment.

Teacher environment
LiMe has been introduced to support the web development for Valaya Alongkorn
Rajabhat University (VRU). This university had attempted to implement an eLearning policy but this did not work in their environment. LiMe was employed to to
help improve the teaching and learning environment.

In order to do this, LiMe started from the teaching environment. It mainly supported
the teachers in the grading system. In this environment, the activities of students andteachers are homework submission, class attendance, online exam, and grading.

The web developer used LiMe to find useful concepts of the teaching environment
from the ontological framework and improved the framework with their teacher
working experiences. This provided suitable design requirements and a clear picture
before developing the software. Teachers had the opportunity to contribute theirrequirements.

67

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

LiMe was especially useful to reduce the time of the requirement collection. The, it
was used to present the environment structure. This structure could be modified or
improved to accommodate the individual requirements. The following figure (Figure4) represents the screen snapshot of the application that used LiMe to design andorganize information from the ontological framework.

Figure 4: web-based application in the teacher ontological framework.

Faculty environment 1
In this experiment, LiMe has been used to develop the information management for
the faculty environment in Thailand. It provided the ontological framework that
represented information about faculty. Figure 5 shows the Faculty of Science andTechnology at VRU. Faculty improved or modified this framework from theirrequirements. LiMe improved the budget management framework for every section in
the faculty.

68

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

LiMe also provided useful features in the operation patterns. It helped define services
for each member in the section as service framework. With LiMe, level of services
not only is classified but also related to the relevant information from the type ofmember of staff using the system.

Figure 5: web-based application in the Faculty environment

In addition, the users described their projects within the ontological framework
developed from the university framework. This helped establish the required
interoperability between the faculties in the university. The process of implementing
university strategies and monitoring project quality assurance was also improved.

Faculty environment 2
In this experiment, an ontological framework has been applied to develop andorganise the information for the inspection system of the Royal Thai Government(Figure 6).

69

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

In Thailand, projects are created from the organisation of Ministries. Many projects
contribute to a budget plan. Projects need to be tracked to make sure they are
implemented correctly also in remote project areas such villages, and provinces. The
inspection serves not only to monitor the processes but also to provide relevant
information to the project owners.
LiMe has been used to collect the requirements and design the model of thegovernment inspection. This model will be deployed in the real environment.
Therefore, these requirements are very important and need a suitable structure of
information to support in the inspection process.

Figure 6: The inspection in Thailand.
Practically, the government inspection has been designed in five processes: plan
direction, plan preparation, investigation, report, and knowledge management. An
ontological framework was developed for organising and retrieving the informationneed in the system. It also helped to classify the project problems collected from70

The IAFOR Journal of Education

various areas.

Volume 1 - Issue 1 - Spring 2013

The common understanding of the projects was provided to theinspectors. This information was stored as the central information sources.

The Plan Direction process is the defining process. Problems coming from previous
projects will be addressed and used to find solutions or improvements. It involves
document classification, risk analysis, and inspection background information. This is
the useful information to support the inspectors from the remote area.

The Plan Preparation process is the plan creation. The inspectors will design the tasks,
problems, and schedules required by the projects. The relevant information about the
remote area, such as contact information, activities, requesting, and others, will beorganized to support the task. The topics that needed further information will be
developed. The most important information is the project details.

The Investigation process is the data collection process. Information has been
captured from the remote areas by the government inspectors. Suggestions andsolutions will be provided. These are the results of the project operations. Feedback
from the projects is stored in the structured information.

The Report process is the results representation. In order to improve the projects, allinformation that captured from the remote areas is provided. Different perspectives of
the information will be developed and also the comments or suggestions will be addedin this process. LiMe applied the ontological framework in the report system. It
provided the report designing for the users which allowed modifying the reporttemplates based on the individual requirements.

71

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Lastly, the Knowledge Management process is the core process that applies to every
process. It involves information classification. Knowledge is the information used tosolve the problem. This is fully supported from LiMe methodology which captured
the information, extracted the concepts, developed the ontological framework, andtranslated it into formal languages.

Basically, the different inspectors introduced different meanings for the information.
This is the feature the ontology approach was most useful with. The common
understanding of the information will be useful to the environment. The accuracy ofaccessing the right information from the existing framework was very useful.
Knowledge itself could be improved from the descriptions. Therefore, this model willbe the more successful, the more members are participating to it. For this reason, thesystem was designed the experiment as a social network (Figure 7), where members
can interact by sharing knowledge, experiences, problems, suggestions, comments,
etc, not only as text but also images, and video clips.

Figure 7: This inspector application designing

72

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Conclusions
Results
This research contributed various terms or methods to this research. For example, theterm “Knowledge Information” was introduced to represent the crucial informationthat is extracted from research articles. This KI has been captured and described basedon the individual perspectives from the researchers. It is very useful information,
especially for the researchers that require the articles related to their research areas.
Instead of searching the academic journal websites, they can use this information to
retrieve the related information and access the articles from the journal websites.

LiMe has classified knowledge into two different kinds: User-defined requirements
and Knowledge Information. The User-defined requirements are knowledge that iscontributed by the ontology developer, or domain expert. Knowledge Information isknowledge that is captured from research articles. LiMe represents knowledge bydeveloping the combination between knowledge in an Ontological Framework.

An Ontological Framework is the intermediate information that provides the
specification of knowledge in the domain. It has been represented with a hierarchy ofconcepts which is called taxonomy. LiMe integrates the taxonomies in the domain ofinterest based on the knowledge topics.

In order to interact with this framework, LiMe proposes the Knowledge Blog to aid
the knowledge representation. It has been developed for retrieving, describing, andanalysing the knowledge from the domain.

73

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Future Works
LiMe has been proposed as an open environment methodology which extends current
methodologies. It is still in the development stage. Therefore, it is available to any
developer wishing to use this methodology to develop any Ontologies in any researchdomain.

To perform at its best, LiMe needs a lot of information about the domain. Moreknowledge information will produce more Metadata to describe the domain.
Therefore, LiMe needs a way to integrate Ontological Frameworks, and it could be
improved by applying results from the ontology community working on Ontology
merging, Ontology mapping, and Ontology alignment methodologies.

LiMe classifies the knowledge based on the individual topics. Therefore, flexible
information will be represented in different ways. Similarity methods are needed toresolve the problems of inconsistency in the Ontological Framework.

In practice, LiMe uses description languages such XML(s), RDF(s), and OWL toshare and reuse an Ontological Framework. The specification of these languages orversioning will enhance the reliability to describe the context and characteristics ofknowledge in Ontological Framework for individual environment domains.

74

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

References
Asanee, K. (2004). Automatic Thai Ontology Construction and Maintenance System.
Workshop on OntoLex LREC conference, Bisbon, Portugal.
Al-Mubaid, H.N. (2006). Using MEDLINE as Standard Corpus for Measuring
Semantic Similarity in the Biomedical Domain. BioInformatics and
BioEngineering, 2006. BIBE 2006. Sixth IEEE Symposium on, 315-318.
Andreasen, T.B., H.; Knappe, R. (2003). Similarity from conceptual relations. Fuzzy
Information Processing Society, 2003. NAFIPS 2003. 22nd International
Conference of the North American, 179-184.
Cooper, P. (2007). Knowledge management. Anaesthesia & Intensive Care Medicine,
8(12), 516-517.
Dai, B.A.W. (2005). Architecture for automated annotation and ontology based
querying of semantic Web resources. Web Intelligence, 2005. Proceedings. The
2005 IEEE/WIC/ACM International Conference, 413-417.
Eriksson, H. (2007). The semantic-document approach to combining documents andontologies. International Journal of Human-Computer Studies, 65(7), 624-639.
Gahegan, W.P.a.M. (2007). Beyond ontologies: Toward situated representations ofscientific knowledge. International Journal of Human-Computer Studies, 65(7)674-688.
Gomez-Perez, A.n., Fernandez-Lopez, M., and Corcho, O. (2004). Ontologicalengineering : with examples from the areas of knowledge management, ecommerce and the semantic Web. Advanced information and knowledge
processing. 2004, London: Springer-Verlag. xii, 403.

75

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Ganjisaffar, Y.A., H.; Neshati, M.; Jamali, M. (2006). A Similarity Measure for
OWL-S Annotated Web Services. Web Intelligence, 2006. WI 2006.
IEEE/WIC/ACM International Conference on, 621-624.
Hwang, M.K., Hyunjang; Baek, Sunkyoung; Hwang, Kwangsu; Kim, Pankoo (2007).
The Techniques for the Ontology-Based Information Retrieval. Advanced
Communication Technology, The 9th International Conference, 2, 1365-1369.
Maria Ruiz-Casado, E.A.a.P.C. (2005). Automatic Extraction of Semantic
Relationships for WordNet by Means of Pattern Learning from Wikipedia. Natural
Language Processing and Information Systems, ed. L.N.i.C. Science. Vol. 3513.
Heidelberg: Springer Berlin 67-69.
M del Puerto Paule Ruiz, M.J.F.D. (2008). Francisco Ortín Soler and Juan Ramón
Pérez Pérez, Adaptation in current e-learning systems. Computer Standards &Interfaces, 30(1-2), 62-70.
Mahalingam, K., and Huhns M. N. (1997). A tool for organizing Web information.
Computer, 30(6), 80-83.
Nenad Stojanovic, L.S. (2002). Searching for the knowledge in the Semantic Web.
American Association for Artificial Intelligence.
Norm Friesen, R.M. (2005). CanCore: best practices for learning object metadata inubiquitous computing environments. Pervasive Computing and Communications
Workshops, 2005. PerCom 2005 Workshops. Third IEEE International
Conference, 317-321.
Brewster, C. A., and O'Hara, K. (2007). Knowledge representation with Ontologies:
Present challenges - Future possibilities. International journal of human computerstudies, 5(7), 563-568.

76

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Patil, L. (2005). Ontology-based exchange of product data semantics. Automation
Science and Engineering, IEEE Transactions on [see also Robotics andAutomation, IEEE Transactions], 2(3), 213-225.
Sheng, H. (2004). Ontology mapping for interaction in agent society. Services
Computing. (SCC 2004). Proceedings. 2004 IEEE International Conference on,
619-622.
Hübner, S. R.S., Visser, U., and Vögele, T. J. (2004). Ontology-based search forinteractive digital maps. Intelligent Systems, IEEE [see also IEEE Intelligent
Systems and Their Applications], 19(3), 80-86.
Chung, W., Bonillas, A., Lai, G., Xi, W., and Chen, H. (2008). Organizing domainspecific information on the Web: An experiment on the Spanish business Web
directory. International Journal of Human-Computer Studies, 66(2), 51-66.
Liu, X. J., Wang, Y.L., and Wang, J. (2006). Towards a Semi-Automatic Ontology
Mapping - An Approach Using Instance Based Learning and Logic Relation
Mining. Artificial Intelligence, 2006. MICAI '06. Fifth Mexican International
Conference, 269-280.
Huang, W. (2003). Contextual knowledge representation, retrieval and interpretationin multimedia e-learning. Information Reuse and Integration, 2003. IRI 2003.
IEEE International Conference, 258-265.
Zhang, P.Q.S. (2006). Ontology Mapping Approach Based on Concept Partial
Relation. Intelligent Control and Automation, 2006. WCICA 2006. The Sixth
World Congress on, 1, 4108-4112.
Zhumin, S.L.M.J.L.L.C. (2006). Fuzzy Similarity from Conceptual Relations.
Services Computing, 2006. APSCC '06. IEEE Asia-Pacific Conference on, 3-10.

77

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  
	
  

78

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Evaluation of Teaching Performance of English Courses by Applying Data
Envelopment Analysis and Two-phase Segmentation
Bernard Montoneri

Author’s Note
The author would like to thank Providence University of the Republic of China(Taiwan) for financially supporting his attendance to the Third Annual Asian
Conference on Education, Osaka, Japan (October 27-30, 2011).

79

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Abstract
Effective teaching performance is a crucial factor contributing to students’ learning
improvement. Students’ ratings of teachers at the end of each semester can indirectly
provide valuable information about teachers’ performance. This paper selects classesof freshmen students taking a course of English in a university of Taiwan from theacademic year 2004 to 2006 as the research object. We adopt the data envelopment
analysis, a reliable and robust evaluation method, to identify the relative efficiencies
of each class. The calculation is performed in two phases. In phase 1, all the classes
are in the same pool. The results of numerical analysis in phase 1 are used to clarify
whether the existing teaching methods can achieve the desired results and what are
the improved methods. Based on the calculation of phase 1, we segment all theclasses into 2 groups according to their contribution of output indicators in calculating
efficiency values. The empirical results are expected to identify more objectiveclasses and to reveal that the evaluated classes refer to different efficient classes indifferent phases and their ranking order changes accordingly. This method can help toprovide some concrete and practical teaching strategies for the inefficient classes.
Keywords: data envelopment analysis; English courses; teaching performance;
segmentation.

80

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Introduction
English remains an indispensable communication tool and a valuable skill for the
English as second language learners who expect to enter the job market. In Asian nonLatin speaking countries such as Taiwan, Japan, China, and South Korea, students
often struggle to have a good command of the English language in their professional
life. Effective teaching performance is a crucial factor contributing to students’learning improvement. Students’ ratings of teachers at the end of each semester can
indirectly provide valuable information about teachers’ performance. Key
performance indicators (KPIs) are measures of accomplishment. Without theevaluation of performance based on key factors and indicators, there will be nopermanent change and improvement in the enhancement of the quality of educational
institutions (Azma, 2010).

This paper randomly selects 25 classes (among around 250 classes) of freshmen
students taking a course of English in a university of Taiwan of the academic year
2004 to 2006 as the research object. We adopt the data envelopment analysis (DEA),a reliable and robust evaluation method, to identify the relative efficiencies of eachclass. This study focuses on four indicators as an example: two inputs (the course isclearly explained and can easily be assimilated and good communication channels
between the teacher and the students) and two outputs (students’ satisfaction about
their grades and students’ learning performance). These four representative indicators
were selected among a total of 10 and have passed the Pearson correlation coefficienttest. The calculation is performed in two phases. In phase 1, all the classes are in thesame pool. The results of numerical analysis in phase 1 are used to clarify whether the
existing teaching methods can achieve the desired results and what are the improved

81The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

methods. Based on the calculation of phase 1, we segment all the classes into 2 groups
according to their contribution of output indicators in calculating efficiency values.
The empirical results are expected to identify more objective classes and to reveal thatthe evaluated classes refer to different efficient classes in different phases and theirranking order changes accordingly.

The remainder of this paper is organized as follows: section 2 (literature review)presents some academic studies in relation with our research. Section 3 (methodology
and selected evaluated indicators) introduces the DEA model, explains the method
used, presents the data and the important indicators discussed in this paper. Section 4(empirical results and suggestions) presents the obtained numerical results based on
the empirical data which include the efficiency analysis and the segmentation
analysis. Section 5 draws the conclusions, limitations and directions of future studies.

Literature review
According to Sanders & Horn (1998), students with comparable achievement levels insecond grade had different outcomes in fifth grade because of a large number ofvariables such as socio-economic status, school, and class size. But the variable which
had the greatest impact on student achievement was teacher quality. Because teacher
performance is so essential to student accomplishment, many studies have tried todefine key performance indicators (KPIs) in order to assess and to improve teacher
performance. KPIs are tools used by individuals and organizations to track progressand success. Milken (2000) developed a teacher performance based accountability
system in public schools in Arizona using indicators such as teacher skills,

82

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

knowledge, and responsibilities, classroom-level student achievement gains, andschool-wide achievement gains.

In 2002, the National Committee for the Evaluation of the University System(CNVSU) organized in Italy an expert team to devise a teaching evaluationquestionnaire, the Short Form Questionnaire (SFQ), to ensure homogenous evaluationin all Italian universities (Iezzi, 2005). The SFQ defined several indicators, such as
the structure of the degree, the organization of the course, didactic activity and study,
infrastructures, and interest and satisfaction.

Loveland and Loveland (2003) discussed a large number of suggestions for improving
the ratings of 10 factors identified as significant such as (in order of priority)
knowledge of the subject, communication skills/ability, enthusiasm for the subject,encouragement of student participation, rapport with students, fairness in grading,timeliness in providing feedback, organization of class, adequacy of text-book andother learning materials, and instructor's preparation for class.

Wolf et al. (2004) described the weaknesses (poor delivery of course contents, being
disorganized, inaccessible, and displaying weak teaching skills) and the qualities(being a knowledgeable and strategic teacher, creating a positive learning
environment, demonstrating professionalism, demonstrating positive personal traits,
and displaying scholarly traits) in faculty teaching performance.

Johnes (2006) applied Data envelopment analysis (DEA) to measure the performanceof Higher Education institutions (HEIs). This study uses an output-oriented approach

83

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

and indicators such as score based on best 3 A levels or equivalent, gender, school, %of graduates who are female, % of graduates who did not attend an independent
school, and pass/other. Johnes (2006) shows that measures of the efficiency ofdepartments derived from individuals’ efficiencies are much more highly correlatedwith department level efficiency scores.

Martin (2006) applied DEA methodology and selected indicators concerning both theteaching and the research activity of the departments of the University of Zaragoza(Spain) in order to assess their performance. The inputs selected were humanresources, financial resources and material resources; the outputs were credits
registered × experimental coefficient, Ph.D. credits offered, Ph.D. completions,annual research incomes, and scientific production index.

McGowan & Graham (2009) highlighted four indicators contributing most to
improved teaching: active/practical learning, teacher/student interactions, clear
expectations/learning outcomes, and faculty preparation.

Wu and Li (2009) constructed a performance measure indicators system for higher
education using four perspectives: financial, customer, internal process, and learning& growth. Zhou and Wang (2009) applied DEA to analyze the efficiency of 16
universities in China. Their performance indicators are teachers as labor power index,
financial power, physical power, number of graduates, and scientific research.

Montoneri et al. (2011) applied DEA to assess the performance of English writing
courses in a university of Taiwan and selected four indicators: preparation of teaching84

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

contents, teaching skills, fair grading, and students’ learning performance. They
showed that the evaluated classes may refer to different facet reference sets according
to their actual values located in lower or higher ranges. As a result, inefficientevaluated classes may compare themselves with efficient evaluated classes in theirrange and make improvement little by little.

Various studies have been conducted on the KPIs of evaluation, but there is little
consensus concerning the choice of indicators to assess the performance of teachersand educational institutions. The main purpose of this research is not to decide whichindicators are the most suitable, but to find the more important indicators and help toformulate improvement suggestions for educators.

Methodology and selected evaluated indicators
The efficiency assessment is often conducted by DEA which can measure the relative
efficiency of educational institutions from commonly available performanceindicators. This paper uses DEA to investigate the indicators contributing to teaching
performance in a university of Taiwan. We use students’ ratings of teachers(questionnaires filled at the end of each semester) about the course they follow.

Origins and application of DEA
The starting point of DEA is attributed to Farrell’s seminal 1957 paper (Førsund andSarafoglou, 2002). In his study, Farrell introduced his concept of efficiency
measurement. This concept became more popular after Charnes, Cooper, and Rhodes(1978) developed Farrell’s efficiency measurement concept. Their method, the socalled “Charnes-Cooper-Rhodes (CCR) model” or “CCR model” includes the85

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

function and concept of benchmarking and introduced the concept of multiple inputs
and multiple outputs. The CCR (ratio) model is nowadays the most widely used DEA
model. If the efficiency value of the CCR model equals 1, the evaluated unit isefficient (of optimal performance); if the efficiency value is less than 1, the evaluatedunit needs some improvement (Lin et al., 2009; Lee, 2009).

DEA is a reliable and robust evaluation method which has notably been applied toassess the efficiency of educational institutions (Ahn et al., 1989; Johnes & Johnes,
1993; Ng & Li, 2000; Abbott and Doucouliagos, 2003; Johnes, 2006; García-Aracil
and Palomares-Montero, 2008). It has also been applied more recently to assess theperformance of various courses (Mathematics and Science in Ismail, 2009; English
writing courses in Montoneri et al., 2011).

DEA modelThis paper adopts the evaluating method—DEA to perform the efficiency evaluationsof a course of English for freshmen from various departments. We investigate therelative efficiency of decision-making units (DMUs), that is, the evaluated classes.
The DMUs’ relative efficiency values are calculated under an output oriented CCR
model. According to Montoneri et al. (2011), minimizing input indicators in order toobtain an efficiency value equal to 1 can mislead educators. Therefore, the output
oriented model is more suitable than an input oriented model, notably because it canemphasize on how much the insufficiency of the output performance is under thecurrent input resources without additional input efforts.

86

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Data selecting—input and output indicators
The data source
The study case is a private university established in 1956 in Taiwan. There areapproximately 11,000 undergraduate students in the university. The data comes fromthe university’s online student rating system, which provides student feedback toprofessors at the end of each semester. Students are required to fill out thequestionnaires.

The characteristics of the research object are as follows:
1. Freshmen students in a university of Taiwan from the academic year 2004 to2006.
2. The classes are randomly selected from around 250 classes among 21
departments. English majors from the Department of English Language, Literatureand Linguistics are not included.
3. English is a required course for freshmen for all the departments of the studied
university. All the classes follow a similar course to meet the homogeneity of theevaluated object.
4. The English course is a 2-credit course (2 hours/week). Each teacher can choose
the text-book of his/her choice. Most of the teachers propose group discussions
and role plays during the class.
5. A total of 25 classes taught by full-time and part-time teachers are selected as thedecision making units (DMUs), that is, the evaluated units. They are named from
D1 to D25.
6. Among the selected departments for this research: Department of Mass
Communication, Department of Law, Department of Chinese Literature,

87

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Department of Social Work and Child Welfare, Department of Applied
Chemistry, Department of International Business, Department of Accounting,
Department of Tourism, Department of Computer Science and Information
Engineering, and Department of Finance.

The characteristics of the data source are as follows:
1. The data are based on questionnaires (10 questions) filled out by the students at
the end of each semester for each class. Each question is rated from 1 (veryunsatisfied) to 5 (very satisfied) by the students.
2. This paper aims at providing a method to identify the indicators contributing toteaching performance; this method can be applicable to different kinds of data andvarious types of courses.
3. To ensure the reliability of the questionnaires, at least half of the class must
answer seriously. If a student gives ratings too different from the rest of the class,he/she is excluded.
4. The average scores of each question undergo a correlation analysis to test thereliability of the ratings and to find representative indicators in this study.
5. The data concerning the selected indicators is fed in the software Frontier Analyst
to calculate the performance values of each evaluated class.
After the rule of thumb, the number of evaluated units is suggested to be twotimes or even four times the number of indicators. Based on the questionnaires, four
indicators are appropriate in the current research. The indicators selected for theevaluation model are abbreviated by I1, I2 and O1, O2 respectively and presented.

88

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Input indicators
I1.

Course clearly explained and easily assimilated: it refers to the degreeof teachers’ professional knowledge for the preparation of the course.
I2.

Good communication channels between the teacher and the students: it

indicates whether the teacher can actively answer students’ queries and clear
their doubts. It signifies whether teachers can adapt to students’ learning habits
and their learning channels. This indicator may increase students’ learninginterest and learning motivation.
Output indicators
O1.

Students’ satisfaction about their grades: students fill the questionnaire

before the end of the semester; therefore this indicator should not represent
students’ immediate response to one particular grade, but a general appreciationof the fairness of grading during the whole semester.
O2.

Students’ learning performance: it indicates students’ self-recognitionof learning performance after receiving a period of language training. This
indicator relates teacher quality to student achievement.

Correlation analysis of input and output indicators
As mentioned in Lin et al. (2009), the Pearson correlation coefficient test is often used
to verify whether the correlation is high among variables. A closer relation between
two variables means that their correlation coefficient is higher, while less correlated
variables have a lower correlation coefficient. Generally speaking, a Pearson
correlation coefficient of 0.8 or above represents a very high correlation; a value of
0.6 to 0.8 represents a high correlation; a value of 0.2 to 0.4 represents a low
correlation; the value inferior to 0.2 represents an extremely low correlation or not

89

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

correlated. The correlation coefficients among the four selected indicators listed inTable 1 below are all above 0.8 with a significant level of 1%. This shows a very highdegree of correlation. The principle of isotonicity is satisfied.
Table 1. Pearson correlation coefficients between input and output indicators.
Inputs
Outputs

I1(Course clearly
explained and easily
assimilated)

O1 (Students’ satisfaction about
0.851*
their grades)
O2(Students’learning
0.925*
performance)
Notes: 1. * denotes significant levels at 1%.

I2(Good communication
channels between
teacher and students)
0.928*
0.936*

Empirical results and suggestions
The 25 DMUs’ relative efficiency values are calculated under an output oriented CCR
model of DEA and are conducted in two phases. In phase 1, all the 25 DMUs are in
the same pool. The results of numerical analysis in phase 1 are used to clarify the
relative efficiency of each DMU and the indicators’ contribution in calculating
efficiency value. In phase 2, the 25 DMUs are segmented according to their output
indicators’ contribution in calculating efficiency value acquired in phase 1. The
purpose of this segmentation is to regroup DMUs of similar characteristics and to
identify the more objective DMUs which are suitable for designing questionnaires
concerning teaching performance evaluation. This study can provide suggestions toteachers about how to make a better use of limited teaching resources in order toincrease their teaching efficiency in short term.

90

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

DMUs’ efficiency analysis in phase 1
Table 2 lists some performance indicators of the DMUs which are ranked bydescending order of “Efficiency value”. The DMUs with an efficiency value equal to
1 are efficient can constitute “reference sets” which form efficiency frontier curves. If
the efficiency value is less than 1, the evaluated unit is inefficient. The efficient
DMUs are the referring standards for other inefficient DMUs. The efficiency value ofeach DMU is calculated by the distance of their locations to the efficiency frontier
curves. The results show that the average efficiency of all the DMUs is 0.968; that of
the inefficient ones is 0.962. The efficiencies of the DMUs D15, D20, D19 and D16
in phase 1 show the best performance with value of 1. That is, they are all on theefficiency frontier curves without the need of further improvement in the inputs andoutputs. The inefficient DMUs can improve their efficiency by referring to theefficient DMUs of their reference set.

The input and output indicators’ contribution in calculating DMUs’ relative efficiency
values gives information about their importance. As a result, the values listed in Table
2 allow us to identify which inputs and outputs have been used or not in determining
efficiency. For example, the contributions of O1 (students’ satisfaction about theirgrades) and O2 (students’ learning performance) in calculating D15’s relative
efficiency values are 71.7% and 28.3%, respectively; and the contribution values of I1(course clearly explained and easily assimilated) and I2 (good communication
channels between the teacher and the students) are 0% and 100%, respectively. Thismeans that for D15, students’ satisfaction about their grades is almost 3 times more
important than students’ learning performance in calculating its relative teachingefficiency, which is only influenced by the input indicator I2; that is, the good

91

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

communication channels between the teacher and the students. The input and output
indicators’ average contributions for all the DMUs reveal that O2 and I2 are the major
indicators in the efficiency evaluation of studied empirical example, with 61.3% and80.9%, respectively. That is, generally speaking, the students’ learning performance isthe major output indicator and the good communication channels between the teacher
and the students is the major input indicator.

Suggestions. In order to improve teaching performance, teachers of inefficient DMUs
should emulate the efficient DMUs of their reference set and focus on enhancing thecommunication channels, adapt to students’ learning habits and their learning
channels, such as language learning websites, learning software, online courses,
mobile phones, Twitter, Facebook, blogs, etc., in order to give them enough learning
support during and outside the class. Consequently, students’ learning motivation andperformance will be increased accordingly.

92

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Table 2 Relative performance indicators of DMUs in phase 1
DMU name
D20
D16
D19
D15
D22
D24
D13
D7
D17
D1
D25
D21
D10
D9
D4
D14
D5
D3
D12
D2
D23
D11
D18
D8
D6

Efficienc Rany value
k
1.000
1.000
1.000
1.000
0.990
0.986
0.985
0.980
0.978
0.975
0.969
0.967
0.963
0.960
0.959
0.959
0.957
0.957
0.956
0.956
0.950
0.950
0.947
0.943
0.920

1
1
1
1
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

Reference
set
D20
D16
D19
D15
D20
D15, D19
D15, D20
D15, D20
D15, D19
D15, D20
D15, D19
D15, D20
D15, D20
D20
D15, D20
D15, D20
D15, D19
D15, D20
D15, D19
D15, D20
D15, D16,
D19
D15, D19
D15, D20
D15, D19
D15, D19

Contribution in calculating efficiency
value (%)
O1
O2
I1
I2
26.1
73.9
0.0
100.0
95.7
4.3
81.6
18.4
74.5
25.5
0.0
100.0
71.7
28.3
0.0
100.0
0.0
100.0
0.0
100.0
72.3
27.7
0.0
100.0
0.0
100.0
12.5
87.5
0.0
100.0
12.1
87.9
72.3
27.7
0.0
100.0
0.0
100.0
12.6
87.4
72.7
27.3
0.0
100.0
0.0
100.0
12.3
87.7
26.4
73.6
0.0
100.0
0.0
100.0
0.0
100.0
0.0
100.0
12.5
87.5
0.0
100.0
12.2
87.8
72.5
27.5
0.0
100.0
0.0
100.0
12.4
87.6
72.1
27.9
0.0
100.0
0.0
100.0
12.3
87.7
96.0
4.0
82.5
17.5
71.7
28.3
0.0
100.0
0.0
100.0
12.6
87.4
72.2
27.8
0.0
100.0
71.9
28.1
0.0
100.0

Average of all
0.968
38.7
61.3
11.0
89.0
the DMUs
Average of theinefficient
0.962
33.3
66.7
9.2
90.8
DMUs
Note: O1 is “students’ satisfaction about their grades”; O2 is “students’ learning
performance”; I1 is “course clearly explained and easily assimilated”; I2 is “goodcommunication channels between the teacher and the students”.
DMUs’ efficiency analysis in phase 2 — Segmentation of DMUs by output
indicators’ contribution
Based on the calculation of phase 1, we segment all the DMUs into 2 groups
according to their output indicators’ contribution in calculating the relative efficiency.
The DMUs with O1’s contribution superior to 50% are classified as the group O1

93

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

which contains 12 DMUs: D16, D19, D15, D24, D17, D25, D5, D12, D23, D11, D8
and D6. The DMUs with O2’s contribution superior to 50% are classified as the group
O2 which contains 13 DMUs: D20, D7, D10, D13, D22, D1, D21, D14, D2, D4, D3,
D9 and D18. For example, D16 belonging to group O1 has O1’s contribution (95.7%)
superior to that of O2 (4.3%).

In phase 2, the calculation of each DMU’s relative efficiency is separately conductedin the two groups and the efficient frontier curves are reconstituted in the twodifferent segmented groups. Table 3 includes each DMU’s relative efficiency, rankorder and output indicators’ contribution in calculating relative efficiency in phase 1
and phase 2. The results reveal that:One new efficient DMU appears in phase 2. The 3 efficient DMUs (D16, D19, andD15) in phase 1 are still efficient in phase 2; but one more DMU (D7) becomesefficient in phase 2 and is located in the segmented group O2. Because thesegmentation according to output indicators’ contribution makes the newreconstituted frontier curves in group O1 now closer to the O1 value and in group O2
now closer to the O2 value, this results in a new efficient DMU appearing in group
O2 in phase 2.

The DMUs of group O1 are more influenced by O1 in phase 2 than in phase 1; theDMUs of group O2 are more influenced by O2 in phase 2 than in phase 1. Thisphenomenon can be proved by the slightly increase or by the same efficiency value inphase 2 than in phase 1.

94

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Inefficient DMUs refer to different efficient DMUs in different phases. Because 3of the 4 efficient DMUs in phase 1 now belong to group O1, one other efficient DMU
belongs to group O2. This implies that after the segmentation, the efficient frontier
curves are recalculated and the efficient DMUs can probably be changed; some of theinefficient DMUs in group O1 originally referring to the efficient DMUs which are
now located in group O2 have to refer to different efficient DMUs, because they are
in different pools. For example, the two inefficient DMUs of group O2, D1 and D21,
originally referred to the efficient DMUs D15 and D20 in phase 1; because D15 islocated in group O1 in phase 2, they refer to the efficient DMUs D20 and D7 instead.

Ranking order changes in different phases. In group O1, the 12 DMUs’ ranking
order in phase 1 is the same as that in phase 2; however, in group O2, the 13 DMUs’ranking order in phase 1 is different from that in phase 2. For example, D22, D1,
D21, D4, and D9 have higher rank in phase 1 than in phase 2; and D7, D10, D14 andD2 have lower rank in phase 1 than in phase 2. Only 4 DMUs in group O2 keep thesame ranking order as in phase 1. There is one new efficient DMU in group O2
because the new frontier curves are closer to O2 in phase 2. Group O2’s efficiency
values are equivalent or slightly higher in phase 2 than in phase 1.

More objective DMUs appear. In group O2, the major indicator of DMUs D7, D10,
D4 and D18 changes from O2 to O1. It implies that these four DMUs are more
influenced by the presence of other DMUs and are less objective concerning theresult of teaching efficiency. As for the DMUs in group O1, their major indicator isstill O1. Therefore, except D7, D10, D4 and D18, all the DMUs of group O1 and O2

95

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

are more suitable for designing questionnaires concerning teaching performance
evaluation.
Table 3. DMUs’ relative efficiency ranks and output indicators’ contribution in twophasesGroup O1Group O2

Contribution
Contributio
Rank
Relative
Rank inof O1 inninefficiency in
Unit
phase
of O2 in
Unit phase
phase
nam phase
phase
name
e
1*
1* 2
1
2
1
2
2
1
2
1
2
*
D16 1 1 1.000 1.000 95.7 100.0 D20 1 1
1.000 1.000 73.9 100
D19 1 1 1.000 1.000 74.5 100.0
D7 4 10.980 1.000 100.0 0
D15 1 1 1.000 1.000 71.7 71.7
D10 7 30.963 0.991 73.6
0
D24 4 4 0.986 0.986 72.3 72.3
D13 3 30.985 0.991 100.0 100
D17 5 5 0.978 0.978 72.3 72.3
D22 2 50.990 0.990 100.0 100
D25 6 6 0.969 0.969 72.7 72.7
D1 5 6
0.975 0.978 100.0 100
D5
7 7 0.957 0.958 72.5 72.5
D21 6 70.967 0.978 100.0 100
D12 8 8 0.956 0.956 72.1 72.1
D14 10 8
0.959 0.973 100.0 100
D23 9 9 0.950 0.950 96.0 96.0
D2 12 90.956 0.967 100.0 100
D11 10 10 0.950 0.950 71.7 71.7
D4 9 10 0.959 0.967 100.0 0
D8 11 11 0.943 0.943 72.2 72.2
D3 11 11 0.957 0.965 100.0 100
D6 12 12 0.920 0.921 71.9 71.9
D9 8 12 0.960 0.960 100.0 100
D18 13 13 0.947 0.951 100.0 0
Note: *: Group O1’s rank in phase 1 means that their new rank does not consider thepresence of group O2’s DMUs. **: Group O2’s rank in phase 1 means that their new
rank does not consider the presence of group O1’s DMUs.
Relative
efficiency in
phase

96

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Conclusions and suggestions
This paper applies DEA to calculate the relative efficiency values of 25 evaluatedclasses under an output oriented CCR model. The calculations are conducted in twophases. In phase 1, all the 25 DMUs are in the same pool. The results are used toclarify the relative efficiency of each DMU and the indicators’ contribution in
calculating efficiency value. All the inefficient DMUs of group O1 (D24, D17, D25,
D5, D12, D23, D11, D8 and D6) are suggested to concentrate teaching effort on
indicator O1 (students’ satisfaction about their grades) in order to increase theirrelative efficiency in short term. Teachers are suggested to announce grading criteria
as clearly and early as possible in order to guide students and to answer theirquestions and doubts before the exams. After the exams, teachers should give a
correction and advices to students. Students who have a bad grade sometimes give up
and drop the class. Under these circumstances, communication channels between theteacher and the students should be fast and clear. Students need to feel that teacherscare about them. In addition, teachers can offer them some help after the class orduring the office hours. Students need to know why they failed, and more important,what they can do to improve their level. This will help enhance students’ learning
motivation and increase the value of O2 (students’ learning performance) at the sametime. All the inefficient DMUs of group O2 (D17, D25, D5, D12, D23, D11, D8 andD6) are suggested to concentrate teaching effort on indicator O2 in order to increase
their relative efficiency in short term. Teachers can offer students help outside theclass (teaching website, English corner, office hours).

In phase 2, the 25 DMUs are segmented according to their output indicators’
contribution in calculating efficiency value acquired in phase 1. The purpose of this97

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

segmentation is to regroup DMUs of similar characteristics and to identify the moreobjective DMUs which are suitable for designing questionnaires concerning teaching
performance evaluation. The analysis of phase 2 shows that except D7, D10, D4 andD18, all other DMUs are more suitable for designing questionnaires. It means that on
25 DMUs, 21 can provide reliable information to educators and decision-makers. The
results may of course vary according to the year, the subject matter, the departments
and the classes selected.

This paper proposes a method to find out the more important evaluated indicators andhelp to formulate improvement suggestions for educators in Taiwan concerning
English courses for freshmen. Our demonstration on how to screen primary indicators
can be useful for further studies in other countries or fields. The results of this paper
can serve as a model for decision-makers to design the educational policies satisfying
the objectives of enhancing the competitiveness of educational institutions. The
results of the study need to be interpreted in light of its limitations. DEA only gives
efficiencies relative to the data considered. This paper offers suggestions to teachers
on how to improve their teaching according to four selected indicators. Future studies
could propose to analyze other indicators and conduct research on teachers’ responseto student ratings.

98

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

References
Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A
data envelopment analysis. Economics of Education Review, 22(1), 89-97.
Ahn, T., Arnold, V., Charnes, A., & Cooper, W. W. (1989). DEA and ratio efficiency
analyses for public institutions of higher learning in Texas. Research in
Governmental and Nonprofit Accounting, 5, 165-185.
Azma, F. (2010). Qualitative Indicators for the evaluation of universities
performance. Procedia Social and Behavioral Sciences, 2(2), 5408–5411.
Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the inefficiency ofdecision making units. European Journal of Operational Research, 2(6), 429444.
Farrell, M. J. (1957). The measurement of productive efficiency. Journal of the Royal
Statistical Society Series A. General, 120(3), 253-282.
Førsund, F. R., & Sarafoglou, N. (2002). On the Origins of Data Envelopment
Analysis. Journal of Productivity Analysis, 17(1-2), 23–40.
García-Aracil, A., & Palomares-Montero, D. (2008). Changes in universities’
efficiency over the time: Differentials according to the missions. First ISA
Forum of Sociology: Sociological Research and Public Debate, Barcelona.
Iezzi, D. F. (2005). A method to measure the quality on teaching evaluation of theuniversity system: The Italian case. Social Indicators Research, 73, 459-477.
Ismail, I. (2009). English in the Teaching of Mathematics and Science Subjects(ETeMS) Policy. Implications for the Performance of Malaysian Secondary
Schools in Mathematics and Science Subjects. 16th EDAMBA Summer
Academy, Soreze, France.

99

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Johnes, G., & Johnes, J. (1993). Measuring the research performance of UK
economics departments: An application of data envelopment analysis. Oxford
Economic Papers, 45, 332–347.
Johnes, J. (2006). Measuring teaching efficiency in higher education: an applicationof data envelopment analysis to economics graduates for UK universities.
European Journal of Operational Research, 174, 443-456.
Lee, C. C. (2009). Analysis of overall technical efficiency, pure technical efficiency
and scale efficiency in the medium-sized audit firms. Expert Systems with
Applications, 36(8), 11156-11171.
Lin, T. T., Lee, C. C., & Chiu, T. F. (2009). Application of DEA in analyzing abank’s operating performance. Expert Systems with Applications, 36 (5), 88838891.
Loveland, K.A., & Loveland, J.P. (2003). Student evaluations of online versus oncampus classes. Journal of Business and Economics Research, 1(4), 1-10.
Martin, E. (2006). Efficiency and Quality in the Current Higher Education Context inEurope: an application of the data envelopment analysis methodology toperformance assessment of departments within the University of Zaragoza.
Quality in Higher Education, 12(1), 57-79.
McGowan, W. R., & Graham, C. R. (2009). Factors contributing to improved
teaching performance. Innovative Higher Education, 34(3), 161-171.
Milken, L. (2000). Teaching As the Opportunity: The Teacher Advancement
Program. Presented at the 2000 National Education Conference President’s
Presentation. Santa Monica, CA: Milken Family Foundation.

100

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Montoneri, B., Lee, C. C., Lin, T. T., &. Huang, S. L. (2011). A Learning
Performance Evaluation with Benchmarking Concept for English Writing
Courses. Expert Systems with Applications, 38(12), 14542-14549.
Ng, Y. C. & Li, S. K. (2000). Measuring the research performance of Chinese higher
education institutions: an application of data envelopment analysis. Education
Economics, 8(2), 139–56.
Sanders, W. L. & Horn, S. P. (1998). Research Findings from the Tennessee ValueAdded Assessment System (TVAAS) Database: Implications for Educational
Evaluation and Research. Journal of Personnel Evaluation in Education, 12(3),247-256.
Wolf, Z. R., Bender, P. J., Beitz, J. M., Wieland, D. M., & Vito, K. O. (2004).
Strengths and weaknesses of faculty teaching performance reported byundergraduate and graduate nursing students: a descriptive study. Journal ofProfessional Nursing, 20(2), 118-128.
Wu, Y., & Li, C. (2009). Research on performance evaluation of higher education
based on the model of BSC-DRF-DEA. 16th International Conference on
Industrial Engineering and Engineering Management. IE&EM '09, 2030-2034.
Zhou, C., & Wang, M. (2009). The Evaluation Research on Higher Education
Efficiency with Data Envelopment Analysis (DEA). International Conference on
Management and Service Science, MASS '09, Beijing, 1-7.

101

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013102

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Practical Consideration of Pair Problem Solving in Computer Literacy
Education
Yoshihiko Oya
Kimiko Uchida

Authors’ Note
This study received financial support from 2008 to 2009 scientific research grants anda Grant-in-Aid© for Scientific Research (Issue Number: 20500816).

103

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Abstract
Direct instruction to students enrolled in a computer literacy program at theundergraduate level frequently involves difficulties due to varied knowledge levels
and skills among the students, as well as an increase in the number of unmotivatedstudents. An available solution is the pair problem solving approach which can prove
to be effective as an effective method. This report shares the findings of aninvestigation regarding the efficacy of pair problem solving, as compared toindividual problem solving in computer literacy education. Furthermore, the paired
approach analysis was able to extract specific criteria for successful pairs. The
research, which included two (paired and individual) 15-minute practicalexaminations and questionnaires, a test on basic scholastic ability, and a survey on PC
experiences, was conducted with approximately 280 students from three universities
who were enrolled in a computer literacy program in 2008 and 2009. The results
reveal that the overall scores of the pairs exceeded those of the individuals. Moreover,more than 90% of students found pair problem solving to be a positive experience.
From the viewpoint of learning effectiveness, it is worth mentioning that the mosteffective pair combinations included those with a small difference in basic academic
ability, a large difference in PC experience, and a partner of the opposite sex.
Keywords: pair problem solving, computer literacy

104

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Introduction
With the advent of declining university enrollments, university instructions are
becoming difficult to be followed because of different cognitive and behavioral
characteristics observed in students, such as lower academic ability and intellectual
curiosity (Figure 1).
The skills needed to operate a computer have diversified and the computer literacy
gap has expanded.
Because of this, there have been arguments for the necessity to strictly review
educational content and methodology particularly for computer literacy education(Murakami et al., 2008). Given the current situation, interactive and participatory
approaches for effective instructions that focus on the student have been taking place.
It has been reported that cooperative learning is very effective in research and inpractice, particularly for pairs and small groups. Therefore, the expectations fromthese methods are increasing (Yasunaga, 2008, Tachibana et al., 2010).

Problems of University
Computer Literacy
Education

Possibility of Pair
Learning Approach

Gap in info rmatio ne ducatio n amo ng H igh
Scho o ls.

Dive rsificatio n in PC
o pe rating skills.

Since pair pro ble m so lving
is e ffe ctive whe n the re are
varying le ve ls o f abilitie s,
it is e ffe ctive in addre ssing
the lite racy gap.

De cre ase in stude nts'
acade mic abilitie s andinte lle ctual curio sity.

Gre ate r co mpute r lite racy
gap at the time o f e nro llingin a unive rsity.

It is po ssible to imple me nt
this appro ach witho ut
affe cting the numbe r o f
stude nts o r te ache rs.

Current Status ofUniversity Education

Atte nuatio n o f stude nts'
se nse o f purpo se andpassive class attitude .

Le arning thro ugh a
traditio nal class appro ach isdifficult.
( tr aditio n al appr o ac h = o n e te ac h e r
te ac h in g all stu de n ts at o n c e )Figure 1 Background of research

105

Acco mmo date s stude nts
ne e ds with a classstructure that fo cuse s o nmutual inte ractio ns o f
le arne rs.

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

The effects of the pair approach within information education suggest possibilities,
such as encouraging information literacy, and stimulating students’ desire to learn,(Takahashi et al., 2004) as well as improving their ability to complete tasks, solve
problems, and learn independently (Terakawa et al., 2005). On the other hand, there
are indications that depending on the pair combination, there may not always be aneffect on learning or that there might be issues with developing methods to form
effective pairs (Kaneko et al., 2007, Takahashi et al., 2010). However, regardless ofthe numerous reports on the subject, there is a lack of understanding of pair
combinations or combination criteria because there are few studies that deal with thisissue. Keeping this in mind, the authors of this study introduced a pair approach into
university computer literacy education in 2008. They examined the effectiveness of
this approach by comparing individual problem solving with pair cooperative problem
solving and verifying the effects pair combinations have on the results. Thus far, it isevident that pair cooperative problem solving improved the overall task achievementlevel and was particularly effective for students with lower grades and with mixedgender pairs (Uchida et al., 2010). The students’ assessment of pair learning was high,indicating that this method was effective in meeting students’ needs (Uchida et al.,2010). However, this method also has certain disadvantages such as striking
differences observed between pair results and either no or negative effects withcertain pair combinations.

This study first reports the problem solving results with pairs from a pair combination
criteria perspective based on the results of pair solving approach in class, conductedfrom 2008 to 2009. It also focuses on the problem-solving process as an index forlearners’ awareness toward working as pairs as well as the quantitative changes in

106

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013utterances among pairs, as a means to examine the issues of problem solving for
selected pairs. Finally, the study considers the pair learning effect from the amount ofutterances and survey results to determine how cooperative problem solving iseffective through conversation and student trends.

Methodology
The subjects of this study were enrolled in a computer literacy program in 3
departments of 2 private universities in Aichi Prefecture. A total of 7 classes and 280
students participated each year for 2008 and 2009. In April, students were surveyed
on pair combination criteria and in July, experimental classes were held for pairtesting (Figure 2).
April

Survey pair composition
basicsApril -June

Classes using traditional
approach (one teacher
teaches all students at once)

Basic acade mic ability surve y
Surve y re garding PC e xpe rie nce

Co mpute r lite racy practical le sso ns

De te rmine pairs rando mly

July

Test classes for pair work

Te st 1 （individually ）15 minute s

Fre e co nve rsatio n
5 minute s

Fre e co nve rsatio n
5 minute s

Te st 1 （in pairs ） 　15 minute s

Te st 2  　（in pairs ） 　15 minute s

Te st 2 （individually ）15 minute s

Que stio nnaire Surve yFigure 2 Outline of the study

107

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Pair Combination Criteria
In 2008, students were surveyed on their basic academic ability, computer experience,
interest in computers, and typing speed in order to gain basic data regarding the pair
combination criteria. Of these four criteria, a prior study has acknowledged therelationship between basic academic ability and scholastic performance of students
after enrolling in university, adapting to university education, and scores in thenational exams. Three other items reflected computer literacy before university,
which is the basic premise for computer literacy education, and were included
because objective data on them is relatively easy to obtain.

Given the results of 2008, the 2009 survey focused on 2 indicators; basic academic
ability, which implied involvement in problem solving and performance in pairs, andcomputer experience before university.

The basic academic ability survey consisted of 20 math and kanji (Japanese character)problems and used an adjusted difficulty level so that performance would
approximate a normal distribution. Math problems were composed of basic mathproblems developed to measure university students’ academic abilities. Kanji
problems referenced the kanji test that measures basic Japanese ability. The survey
lasted 20 minutes and surveys were collected individually for each participant.

The survey on computer experience before university had 20 multiple-choicequestions about the Internet, software, and computer usage inside and outside theschool. In the 2008 survey, there were few questions and the multiple choice answers
varied based on the question. The 2009 survey improved on these two issues. The

108

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

survey time lasted 5 minutes and surveys were collected individually for eachparticipant.

Pair Problem Solving
After 8–10 practical computer literacy classes, students were tested (Test 1 and Test
2) individually and in pairs for 15 minutes (22 questions) based on word-processing
proficiency. Pair groupings were randomly selected to determine the effect of pair
combination criteria. Then, students in each department were divided without bias perclass. Approximately half of the students took Test 1 individually followed by Test 2
in pairs. The remainder of the class took Test 1 in pairs followed by Test 2
individually. In each of the divided groups, almost all students were in the same yearof school and from the same academic discipline. Since one teacher taught the samematerial to both groups, the difference between the groups is presumed to be
negligible. During the test, students solved problems in pairs and individually, and theresults were collected individually for each participant. According to preliminary
investigation, the dispersion for Test 1 and Test 2 was set to a certain level adjusting
the difficulty level so that the average variance of correct responses differed by 15 to
20 percent. Furthermore, in order to eliminate the issues with testing order in 2009
and 2008, the tests were conducted in the reverse order (switching Test 1 and Test 2).

Before the pair test, students were given five minutes for free conversation to develop
smooth communication for each pair’s first encounter. Twenty minutes of
conversation was recorded from the time free conversation began to the end of thepair test. After the test, the students took a survey about their method of problem
solving in pairs. The 2009 survey improved upon the issues with multiple choice109

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013expressions that were apparent during the 2008 study. The survey time lasted 5
minutes and surveys were collected individually for each participant.

Analysis of Results
The analysis of the results employed a standard deviation as a standardized score tocomparatively examine the values from Test 1, Test 2, basic academic ability, andcomputer experience. The amount of utterance was determined by converting theconversations recorded during the pair tests into text. The number of times students
spoke was treated as the amount of utterances and the number of characters wastreated as the utterance character count. The analysis of the pair results used in thisstudy consists of the values that were calculated by subtracting the individual testscores (standard deviation) from each subject’s pair test scores (standard deviation),
added according to pairs.

Results and Interpretation
OutlineLooking at the results from the individual and pair practical tests 1 and 2, the pairtests (average standard deviation: 50.65 in 2008 and 51.62 in 2009) surpassed thescores from the individual test (average standard deviation: 49.34 in 2008 and 48.36
in 2009) (ρ = 0.0015 for 2008 and ρ = 0.0001 for 2009). As an overall trend, thisindicates that the task achievement level improves through pair problem solving.
However, from an individual perspective, there was either no difference between thepair and individual results or the pair results were negative for close to 40% of
students. On examining the relationship between pair and individual tests, trends wereindicated in which pair problem solving had relatively less effect for students who

110

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

scored high in the individual test, while students who scored lower improved (Figure3). Previous research has also extrapolated that working in pairs is more effective forstudents

with

lower

40

grades.
y = -0.5658x + 29.643
R² = 0.3657

30

Pair Results

20

10
0
0

10

20

30

40

50

60

70

80

t value

 　 　ρ

-10
-20
-30

Individual Scores

Figure 3 Individual scores and pair results
Criteria for Pair Combination
Table 1

Pair Com bination
Criteria

Pair Results and Pair Combination Criteria

H （High Pair Results ） L （Low Pair Results ）
 　 　ｎ

 　 　m

 　 　s.d.

 　 　m 　 　s.d.

Basic Academ ic
Ability Difference

92

8.85

6.24

36 14.45

8.29

3. 63

0. 001 * *Difference in PC
Experience

92

8.88

6.81

36

6.86

1. 26

0. 211

 　 　ｎ

7.17

* * ρ<. 01

As for the criteria for pair combination, analysis was conducted for two indicators
suggested to be effective in the 2008 study, computer experience before university
and basic academic ability (Table 1). Group H with a pair score above +10 and GroupL with a score below −10 were selected in order to examine the characteristics of the111

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013pair learning effect. A comparison of groups H and L indicated that the basic
academic ability gap was small. The reason for this is that the gap in basic academic
ability reflects the level of high school that students came from, the academic
discipline, desire to learn, and class attitude. It is possible that these disparities affectthe amount and quality of communication in pair testing. On the other hand, trends
indicated that the gap in computer experience was greater in Group H and lesser inGroup L, although the difference between the two was insignificant. The idea was thatstudents with richer experience taught the students who lacked experience, which
made the pairs more effective. However, the hypothesis is that for pairs with a lower
computer experience, students would get stuck or need help in the same places, andalthough they consulted each other, they could not solve the problems.
Table 2 Pair Results by Gender and Amount of Conversational Utterance

Pair Results

Amount ofConversational
Utterance

same

0.85

2138.3

mixed

6.3

2114.8same

1.68

2200.1

mixed

4.3

2127.4

Pair Gender

Male

FemaleIn addition to the two indicators—basic academic ability and computer experience
before university—it was clear that gender was a factor in problem solving andperformance. Males uttered less overall and male gender pairs were less effective,
while mixed-gender pairs were more effective (Table 2). On the other hand, females
overall were more vocal, although the result was that females vocalized more withsame gender pairs as opposed to mixed-gender pairs. However, females achieved
greater results with mixed-gender pairs as opposed to same gender pairs. Furthermore,

112

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

there was a high correlation between the amount of utterances and the pair resultswith females than with males. The outcome determined that mixed-gender pairs are
more effective, followed by female pairs with male pairs being the least effective.

From the above results, it can be concluded that the most effective pair combinations
have a small gap in basic academic ability, a large gap in computer experience, and apartner of the opposite sex.

Pair Learning Effect and the Amount of Utterances
The vocal data (roughly 100 per year) collected during the pair test was converted
into text. The conversation was analyzed by the amount of utterances and thecharacter count of the utterance.

There was a strong correlation (r = 0.98, y = 19.3x) between the amount of utterances
and utterance character count. The average number of times students uttered during
the 15 minute, 22 question (Q1–Q22) pair test was 106.0 and the average utterance
character count was 2107. In other words, it was evident that there were 7
conversational exchanges every minute and they spoke roughly 20 characters at atime. Moreover, depending on each pair, the utterance character count wasdisproportionate (highest was 4733 characters and lowest was 83 characters) and there
was a large difference between the test results. Examination of the relationshipbetween the overall utterance and pair results showed that vocal pairs were more
effective (Figure 4, r = 0.42).

113

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Looking at utterances for each question, there was more utterance for Q2 (insert apage number in the center of footer) in Test 2, which had a character count of 342,
than Q9 (create an autoshape, and insert characters) in Test 1, which had a charactercount of 188. From these results, we can conclude that depending on the pair, there
was a communication gap and a significant increase in utterances for problems withfunctions including a lot of steps or functions that were used less frequently during
the class.

Looking at the changes in utterances over time, utterances increased in the latter halfof Test 1, which had a higher average score, and the utterance was particularly highfor Q13–Q18. In contrast, Test 2 had higher utterances for Q1–Q11 with significant
reduction in the latter half. Furthermore, there was a difference between Test 1 andTest 2 for the pair learning effect by problem.

30

y = 0.0687x - 3.427
R² = 0.17767

Pair Results

20
10
0
0

50

100

150

200

250

300

350

400

-10
-20
-30

Amount of Verbal Communication

Figure 4 Relationship between pair results and amount of conversational utterance

114

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 201330

Pair Results

20
10
0

-10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Q No.

-20
-30

Figure 5 Pair results by each question（● –Test1 ○ –Test2）

While it was more effective in questions Q13–Q18 for Test 1, Test 2 indicatednegative values for Q16–Q21, which was lower than individual scores (Figure 5).
Compared to Test 1, the difficulty level for Test 2 was slightly higher. This led
students to spend more time communicating during the pair test, leaving less time forthem to solve problems in the latter half of the test.

As demonstrated above, the amount of utterances changed during problem solving foreach pair depending on the difficulty level of the problem and their time management
skills, suggesting that it impacted the positive effect of working in pairs.

Learner Awareness for the Pair Test
Judging from the results of the survey conducted after the pair test, a relationshipbetween the effectiveness of pairs and a trend toward awareness of the pair test wasconsidered. In 2008, the survey included 10 items in 2008, whereas it comprised 11
items in 2009.

115

The IAFOR Journal of Education

Table 3

Volume 1 - Issue 1 - Spring 2013

Survey Items with Significant Differences from Pair Results
Survey Item

Pair Result

ｎ

m

H

174

2.41

L

86

1.99

H

174

2.54

L

86

2.25

H

98

2.33

L

42

2.66

H

76

0.28

L

44

0.07

H

174

2.69

L

86

2.52

Easier to solve as a pair than individually

Consulted during the pair test

A free conversation time before the pair testis necessary

There was sufficient time for the pair test

Communication during the pair test was usefulρJudgement

0.0001**0.0017**0.0028**0.0053**0.0174*

Analysis of the significant difference between Group H, which was highly effective interms of the pair learning effect, and Group L, which was less effective, wasconducted with respect to these questionnaire items. The results of the common items
from 2008 and 2009 were totaled together.

First, Table 3 shows the items that pointed the significant differences. These results
infer a willingness to solve problems cooperatively and communicate with each other,and whether or not they had sufficient time determined how effective pair learningwas. As such, a positive attitude and increasing participation awareness of cooperative
problem solving, expanding the ability to communicate, and improving time
management skills are essential to promoting effective pairs.

116

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

We can interpret from the survey items (Table 4) where there was no significant
difference between confidence in the class and students’ interest toward computers,
and these items are unrelated to the effect. Free conversation time beforehand, the pairtesting evaluation, and students’ interactions that were high across the board, areuseful suggestions for setting up pair approach classes.

Table 4

Survey Items with No Significant Difference with Pair Results
Survey Item

Pair Result

ｎ

m

H

76

1.47

L

44

1.41

H

98

2.72

L

42

2.66

H

98

2.65

L

42

2.60

H

76

1.82

L

44

1.72

H

98

2.50

L

42

2.60

Pair works were easy to understandρ

0.3193

Free conversation time was sufficient

0.2808Pair work approach works well0.2677

Interest in computers

0.2673

Friendly interaction with partner

0.2096

117

Judgement

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Conclusion
The results from the two-year experimental classes with pair testing provided thefollowing findings within computer literacy education at university.
1) Pair problem solving was higher than individual problem solving and it confirmedthat pair task achievement was higher overall. On the other hand, from anindividual perspective, working in pairs was ineffective or less effective for nearly
40% students.
2) The study inferred that the combination of criteria such as mixed-gender pairs withsimilar academic ability and differing computer experience was highly effective.
3) The study discovered characteristics such as a greater discrepancy in the amount ofutterances for certain pairs and remarkable increase in utterances for questions
involving functions with more process steps or functions that were used lessfrequently in class.
4) Amount of utterances changed depending on the difficulty level of the problem ortheir time management skills, indicating an impact on the effect of working inpairs.
5) The study suggested that it is possible to improve pair learning results byimproving students’ participation awareness and positive attitude toward
cooperative learning, as well as improving their ability to communicate and time
management skills.

Further detailed analysis of the issues related to the pair learning approach will be
conducted to resolve factors that affect the positive effect of pair learning. In addition,
this study captured the pair learning effect through short-term experimental classes
and consideration of further long-term application is necessary.

118

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

References
Kaneko, D., Noboriguti ,Y., & Komatsugawa, H. (2007). A practice of using thesystem for group learning. The Journal of Information and Systems in
Education, 22, 85-89 (in Japanese).
Murakami, H., Yamasaki, H., & Terada, Y. (2008). An inspection of skills of acomputer and a literacy education for freshmen. IEICE Technical Report. 108(88), 1-8 (in Japanese).
Takahashi, K., & Shintani, K. (2004). Study of cooperative learning which values
student's communications. Journal of Graduate School of Policy andManagement. Doshisha University, 6, 53-62 (in Japanese).
Tachibana, H., & Fujimura, N. (2010). Knowledge Integration Through Collaborative
Problem Solving in High School Students －Others as Partners in CoConstructing Knowledge. Japanese Journal of Educational Psychology, 58, 111 (in Japanese).
Takahashi, Y., Yamashita, A., & Kawamura, M. (2010). A Study of Home Economics
Classes Using Paired Learning, Journal of Integrated Center for Clinical and
Educational Practice, Saitama University, 9, 111-120 (in Japanese).
Terakawa, K., & Kita, H. (2005). Paired Learning in Programming Education. Forumon Information Technology, 4, 345-346 (in Japanese).
Uchida, K., & Oya, Y. (2010). Conversation Characteristics in Paired Learning at
Computer Literacy Education. Forum on Information Technology 2010, 649650 (in Japanese).

119The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Yasunaga, S. (2008). Classroom Reform in Higher Education －A Review of the Useof Cooperative Learning. Annual Report of Educational Psychology in Japan,
48, 163-172 (in Japanese).

120

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  

121

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  

122

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Reflections on the Final Year Learning Experience –Designing a Capstone Experience
Keith Thomas
Kin Chi Wong
Yi Ching Li
Ching Yan Hung

	
  

123

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Abstract
Cross-sector educational reform to be implemented in 2012 in Hong Kong (HK) is intended
principally to prepare students for the future workplace. One of the explicit requirements for the
new four-year undergraduate curriculum is the inclusion of a capstone course for final year
students. This paper explores the uptake and reported effect of the capstone- liked final year
project using participating students’ experience (voice) in existing undergraduate study
programmes in the Chinese University of Hong Kong (CUHK). Semi-structured interviews
were used to collect student feedback; findings revealed considerations in terms of the core
design elements highlighted in literature. The paper highlights students’ lack of readiness to
successful transition into the workplace, linked to the current academic focus of projects. A
framework that includes learning activities preferred by students is proposed for the final year
learning experience. Findings from this study will be useful for curriculum development andevaluation of the final-year curriculum.
Keywords: cross-sector educational reform, future workplace, capstone course

	
  

124

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Introduction
Hong Kong’s 3+3+4 education reform that introduced cross-sectorchanges to both thesecondary and undergraduate (Ug) systems, has several important goals. These include
increased exposure to non-academic learning experiences; expanding whole person capacity;
supporting a close linkage to workplace; increasing students’ adaptability given rapid changes
in society; and preparing students for a knowledge-based society (Education Commission,2000). This reform, commenced in 2009 at the secondary school level, will extend in 2012 into
the tertiary sector with universities introducing a normative 4-year undergraduate curriculum.
One of the explicit requirements for this new curriculum is the inclusion of a capstone course
for final year students. These dramatic changes in an academic structure require a careful lookat the current curriculum, especially on students’ final year, when students can expect to leave arelatively safe and comfortable environment and move into the workplace. In this newenvironment, as The Higher Education Academy (2006) noted in relation to learning andemployability, performance in disciplinary subjects is not a crucial factor, rather it is capacity
and achievement in a range of soft skills (such as interpersonal skills, communication skills, andpresentation skills etc.) that will most impress potential employers.

This paper reviews the final year project (FYP), the de-facto capstone- type course in a local
Hong Kong university. The aim, using the students’ voice, is to understand if the FYP
implemented as a capstone in the new curriculum will support the development of desired
attributes necessary to support graduate employability. Consistent with an outcome-based
approach to education, the focus should be on authentic learning opportunities in the final year
in order that students may exercise and enhance their soft skills before stepping into future
careers. In other words, higher education institutions should not simply produce disciplinebased outstanding graduates, but a multi-faceted graduate able to meet the needs of the society.

	
  

125

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Education reform in HK
According to the Reform proposals for the Education System in Hong Kong (Education
Commission, 2000), Hong Kong education reform started in late 1990s with the goal ofpromoting lifelong learning and all-round student development. Consistent with, a public
consultation process it was initiated to determine the objectives for education in the 21stCentury for Hong Kong. The results showed that higher education should facilitate students’learning, develop their abilities in effective communication and expand their capacity forcreativity and sense of commitment to their communities. Moreover, in any fast-changing
society, the requirement of multi-faceted talents was favored over specialized talents.
Reflecting these concerns, a report by University Grants Committee (UGC) a nongovernmental body that advises the Government of Hong Kong on the development andfunding needs of higher education institutions (HEI), noted institutions should provide students
with interdisciplinary learning experiences that can equip them with an expanded scope ofknowledge and foresight for a globalised society (UGC, 2010). The subsequent comprehensive
cross-sector educational reform process initiated in 2009 in secondary schools and extending in2012 to the Ug sector echoes these broad objectives for higher education.

New Ug curriculum and the capstone course
The Chinese University of Hong Kong (CUHK), where this study is situated, is one of thepremier research-intensive institutions in Asia. As advocated by the UGC, an outcomes based
approach has been integrated into the design of the new curriculum. Another feature of the new
curriculum is a common Faculty Package for first-year students, with core components of thecurriculum strengthened by the inclusion of General Education, languages, information
technology and physical education units. Another feature of the four-year curriculum,
consistent also with the objectives of the education reform, is the systematic inclusion of a

	
  

126

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

capstone course as the culmination of the undergraduate experience (CUHK, 2011). Different
capstone experiences are being designed to suit the nature of each discipline, however, as the
new four- year curriculum states, the capstone course targets the synthesis of subject
knowledge, as well as independent enquiry (e.g. research) or execution (e.g. creative design inFine Arts or Architecture, engineering design, fieldwork or internship both involving reflectionand evaluation) (CUHK, 2011).

As the University has noted, previous experience and pilot courses with Final Year
Projects have demonstrated benefit in terms of the development of student capabilities. At
present, CUHK has eight faculties offering 62 undergraduate programmes; of these sometwenty- four programmes have a compulsory FYP as a graduation requirement and thirty-eightprogrammes offer a FYP in the form of an elective. As a note of caution, however, it is worth
noting that the majority of FYPs are operated as academic research.

The Capstone Experience. In general a capstone course is intended to integrate a body ofrelatively fragmented knowledge into a unified whole (Atchison, 1993; Durel, 1993). This
integrating activity, allows students the opportunity to look back or reflect over theirundergraduate curriculum in an effort to make sense of that experience. It should also allow
students to look forward in order to transition into working life by building on that experience(Durel, 1993; Henscheid, 2008).

The issue of looking forward is crucial given the realization in the 1970s and 1980s of a
gap between academic study and the real world (workplace). As a consequence, someuniversities moved to develop a course that could bridge this gap (Schroetter & Wendler, 2008).
The resultant course(s) evolved into what is referred to today as a capstone course and that

	
  

127

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

some describe as the “crowning achievement” in an undergraduate programme (Atchison, 1993,
cited in Schroetter & Wendler, 2008). The National Survey of Senior Seminars and Capstone, a
study in the United States, recorded the importance, even critical nature of this course(Chickering and Schlossberg, 1998; Henscheid & Barnicoat, 2001). As this study also noted, it
is often difficult for students to leave their comfort zones and move into a new environment,
and educators need to make an effort to help students move on after graduation.

Chickering and Schlossberg (1998) reported three issues for educators assisting students tosuccessful transition: first, make a career connection, second, help them identify their new roles
after university; and third, create a life-long perspective. Educators should treat this facilitativerole as equally important as helping student transition into university as freshmen. Progressive
design features of a transition-focused capstone include a foundation component, needed toprovide student basic knowledge and skills. These foundations are provided by the formative
courses students complete in the first three years of university study. A second component iswhat can be termed as a pre-capstone component completed towards the end of year 3 and the
beginning of year 4, the capstone year. This component is intended to help student learn
advanced research techniques and like skills in preparation for their final year study. The final
component is the actual capstone course, that some also describe as an ‘experience’ inrecognition that the capstone objectives are likely to be satisfied better by a composite range ofactivities (Hauhart & Grahe, 2010). Reflecting upon the diverse needs of the student body andthe transition needs, as one institution (Copenhaver, 2011) has determined, the capstone
experience is made up of a varied set of options so that students are able to choose theirpersonal capstone experience according to their abilities and future needs.

	
  

128

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Design Characteristics. Literature identifies two broad types of capstone, a developmental
capstone and an assessment capstone. In this paper, the focus of the capstone course isdevelopmental, because this form of capstone tends to be the common approach in higher
education. Reflecting on this focus, there are four broad design characteristic of a capstone
activity or course. These are:•

To encourage integration and synthesis of previously acquired knowledge and skills(Bailey, Oliver & Townsend, 2007; Cuseo, 1998; Jervis & Hartley, 2005). Other researchers
state the integrative focus as students being given a chance to make connections between course
content, acquired skills and application in a wider context (Holdworth, Watty & Davies, 2009;
Huber & Hutchings, 2004; Rowles, Koch, Hundley, & Hamilton, 2004).
•

To facilitate some form of transition, such as from university to professional/working

life (Bailey et al., 2007; Cuseo, 1998; Henscheid, 2000; Schroetter & Wendler, 2008; Wood,2007). This characteristic includes the encouragement of useful connections between study
majors and work experiences, such as those acquired via internships and exchanges; anawareness of personal development necessary to transition from undergraduate to post
university life; and preparation for career or postgraduate education through professional
development (Henscheid, 2000; Jervis & Hartley, 2005). Importantly, as Rosenberry & Vicker(2006) noted, when capstone activities address career issues, students are reported to have abetter understanding of the relevance of what they have learnt and how it can be applied.
•

To assist students to reflect on and demonstrate on what they have learnt over theirundergraduate studies (Holdsworth et al., 2009; Kerka, 2001). Reflective practice is afundamental skill of life-long learners and being able to reflect on one’s performance can also
help achieve higher goals. Hence, reflection is a vital component of the capstone experience(Kift, Field & Wells, 2008) that involves both course content in their academic major and moregenerally across courses, as well as an inner, personal reflection by students on their aims,

	
  

129

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

personal strengths and future plans (Brooks, Benton- Kupper & Slayton, 2004; Henscheid,
2008).
•

Finally, being placed in the final year of an undergraduate degree, a capstone activity

represents a culminating experience (Holdsworth et al., 2009) that arguably offers students achance for closure (Rowles et al, 2004; Schrotter & Wendler, 2008; Schubert, 2009). This is thelast opportunity to ensure students graduate with the knowledge, skills and attitudes they needto meet the growing demands of professional practice (Rowles et al., 2004). The process ofclosure, which includes recognition of accomplishments, pulls together all the ideas presented
in different units and helps construct some sorts of integrated, meaningful whole experience(Heinemann, 1997).
Figure 1 below illustrates a developmental capstone designed to support generic skills and highlevel thinking applicable in the workplace. The central learning outcome of this course
therefore is encapsulated by graduate competence. The focus for students is not about acquiring
new knowledge, but about integrating, reflecting and extending knowledge that has alreadybeen acquired (Bailey et al, 2007; Cuseo, 1998).

Fig. 1 A conceptual illustration of a developmental capstone

	
  

130

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Methodology
The final year project (FYP), a de-facto capstone in the current three-year curriculum was
examined in order to understand the difference, if any, between current practice and the ideal
capstone design experience. The FYP is significant to a student’s final year because, for somedisciplines, it was a compulsory course and students were expected to devote most of their time
to complete this project as it is regarded as an important milestone in their undergraduate
studies.

This investigation adopted a qualitative approach, because it provides a rich pathway to collectinsights and practices through interviews and personal conversations (Brewerton & Millward,
2006; Heppner & Heppner, 2004). As suggested by Brewerton and Millward (2006), aqualitative approach using semi-structured interviews allowed interviewees to offer their own
experiences in a fluid and unrestricted manner, still within the context of the target researcharea. The research framework proposed for this study is based on the earlier defined four design
characteristics for a capstone experience, and a survey done with 300+ graduates.
Eighteen students from thirteen programmes across the Science, Engineering and Humanities
disciplines participated in interviews on a voluntary basis. During the interview, students wereasked if the FYP helped them in developing their future and whether the FYP completed met
the four capstone characteristics evidenced in literature. Participants were also encouraged to
articulate their expectations and opinions on a capstone experience. Interviews were recorded
and then analyzed using thematic analysis, with significant comments and expectations among
students sorted by supporting argument.

	
  

131

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Reported findings
Relative importance of the design characteristics
Based on accumulated responses, all participants agreed that the four characteristics are
essential to the final year of study. However, among the four characteristics, transition received
most attention. Students said it is important to help them understand the workplaceenvironment, provide chances to put theories into practice, and better determine a career path.
At the final stage of undergraduate life, students suggested that knowing specific workplacerequirements and preparing to face unexpected real life challenges were the important reasonsfor a “transition” component in the final year. Students also acknowledged knowing the realworld practice as necessary in order for a smooth transition. For example, a Science student,
who spends 16 hours a day in the library to study, expected he would face many problems in
effective communication when he applies for jobs. According to him, if he had been given achance to know the workplace environment and the relationship between Science and theoutside world, then this would have been most helpful to his future employment.

Another factor highlighted by students as supporting better transition, was the chance to executeand apply what they have learnt in class in their final year. Application of integrated knowledge
serves as an experience for students to evaluate their performance and abilities to manage
hands-on tasks. As several students mentioned:-‘[the opportunity to] use acquired theories to apply into the real world, after suchpractical use of theories, I will never forget these theories, because they transformed into myexperience’ (Humanities student).
-‘Something you may only have came across in one course, then forget, but by doing

FYP you recall your memory and apply it.’ ‘Because you can apply what you have learnt [in

	
  

132

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

final year] and after graduation, you can perform better transition to the real world, you won’t
get lost after graduation’ (Science student).

It is clear that application of knowledge is important to students and for this reason, authentic
assessment is also a crucial element of the study experience. Authentic assessment involves
worthy intellectual challenges, presented as an array of tasks that are likely to be encountered in
the workplace (problem identification, research, analysis, problem resolution and presentation)
which requires full application of acquired knowledge in realistic settings. To illustrate, awritten test, for example, is not a useful basis to infer driving ability. An authentic test would
include some demonstration of ability, as well as an opportunity to improve performance. In all,students endorsed the importance of a final year learning experience based on the four design
characteristics, with a particular emphasis on transition. However it is problematic to discoverwhether students found their FYP experience actually satisfied by these four characteristics.
Evaluation of the FYP
Table 1: Summary of student voice by development theme
Integrati
on (+ve)

Integration(-ve)

Transiti
on
(+ve)

	
  

1.Curriculum design
My discipline emphasizes fieldwork a lot in any course-based projects àcanintegrate the skills of fieldwork methodology.
Have to use previous knowledge in order to produce something new à canalso apply what I have learnt in my minor
FYP reminds me of what I have learnt in year 1 àapply them in the project.
FYP cannot integrate previous knowledge.
Many necessary materials are new
FYP topic is not related to year 1 & 2 courses.
New theories àfeel like taking an extra course rather than consolidating
previous knowledge
2. Focus of the FYP
Cannot prepare me for workplace, but postgraduate study Compulsory
placement demonstrates my competence

133

The IAFOR Journal of Education

Transition (- ve)

Volume 1 - Issue 1 - Spring 2013

Not useful ànot planning to stay in the academic world.
Wonder how an academic thesis is considered to be helpful in a business world.
FYP no use for job applicationàjust a 6-credit course
Research type FYP cannot help job application
Cannot support my transition to workplace, it is too concentrated àcannot
learn anything from other fields.
Won’t stay in the same field àFYP is unhelpful

Reflection andClosure(+ve)

3. The graduate capabilities
Know how to conduct a study effectively and efficiently. Learnt different
theories and became more capable.
Presentation skills become better because of there are chances to practicethroughout the FYP.

Of the four design characteristics identified in literature, two characteristics, reflection
and closure, were reported as being commonly evident in the FYP. However, integration andtransition were not commonly reported. Examining the student feedback, themes related tointegration are categorized under curriculum design, while comments related to transition
aspects are categorized in terms of the focus of capstone. Comments linked to reflection andclosure is grouped within the broad theme of developing graduate capabilities. The following
discussion on student voice is in terms of these three categories (Table 1 is a summary of key
points noted by participants).

Curriculum design. Curriculum design is crucial to a successful implementation of a capstone
experience. The study before the final year, i.e. the foundation and pre-capstone, is also criticalfor preparing the student to reflect and integrate during their final year study. According to thestudent voice, there are both positive and negative feedback on the FYP. Most positive
feedback was gained from the closure and reflection aspects. Student reported that they could
reflect on their performance in soft-skills and personal goals through FYP.
-	  ‘I know how to conduct a study effectively and efficiently’ (Humanities student).

134

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

A science student said because her FYP is to create a new product, so she needed to use her
previous knowledge to serve as the base of reference:-‘I have to use previous knowledge in my FYP, in order to produce something new [anewflavor candy]. I can also apply what I have learnt in my minor [marketing studies], as I
have to conduct marketing research for my new product’ (Student from Science).
-‘FYP reminds me of what I have learnt in year 1, for example some formula, and I

manage to apply them in the project’ (Student from Engineering).
However, this experience was not a common one among other students. More commonly,
students reported that only a limited amount of knowledge was useful to their FYP. Thesestudents, therefore, thought that the FYP did not help them to integrate what they have learnt intheir University life.
-‘This topic is from a year 3 course [final year course], not quite related to year 1 & 2

courses’ (Science student).
-‘I don’t’ think FYP can integrate previous knowledge, in these 3 years, only one course

talks about cultural conservation. Many necessary materials are not being mentioned inprevious years; they are new to me and I have to find them by myself’ (Humanities student).
-‘Many of them are new theories; it seems like taking an extra course rather thanconsolidating previous knowledge’ (Engineering student).
Notwithstanding the fact that students in the FYP could not consolidate what was otherwise
informative knowledge, students found that research skills learned in previous years were
essential to their FYP. As one humanities student noted:-‘My discipline emphasizes fieldwork a lot in course-based projects, so in my FYP, I can

integrate the skills of fieldwork methodology.’
Based on this feedback, it is reasonable to infer that curriculum design and learning activity areimportant when implementing a capstone. To illustrate, without an adequate foundation a

	
  

135

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

capstone experience would become meaningless. Similarly, without authentic assessment, acourse or project would not help consolidate desired capabilities and confirm the ability tosuccessfully apply acquired knowledge in realistic settings, as well as afford some opportunity
to improve performance.

Focus of the FYP. The focus of the FYP affects the reported learning outcomes of students.
Based on student feedback, the current approach to FYPs appears to be academic in –focus andas such students found the experience unable to cater to their needs and abilities. Conversely,
students who were interested in postgraduate studies reported the FYP as most beneficial totheir transition.
-‘I have learnt different theories, more algorithms and [am] more capable of doing[computer] programming’ (Science student).
-‘[the] FYP cannot prepare me for [the] workplace, but it prepares me for postgraduate

study. For example, I know how to do research and [have] discover[ed] that being a MPhil
student is to do research consistently, then write a thesis’ (Science student).
These students stated that the FYP helped them understand the process of academic study. In
this case, the FYP gave final year students a sense of closure and offered them a chance toreflect on what they have learned during their university life. When asked about whether the
FYP helped them integrate previous knowledge and skills, most students mentioned integrationin terms of research skills, but not the consolidation of knowledge and skills. One issue withintegration was that students noted the courses offered in the previous years were too diverse.
Another issue was, given that the FYP was focused on academic research and so narrowed
down to a specific topic, the approach allowed very limited inferences from previous studies.
The sum effect is that the focus of the FYP is academic research and as a result viewed asoffering limited opportunity for programme-level knowledge integration. Moreover, with this

	
  

136

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

academic focus, the FYP also offers little opportunity for the development of soft skills, such as
independent problem solving, self-management, communication and teamwork. These andother workplace capabilities, including technological awareness and initiative and enterprise areimportant graduate capabilities (Kember & Leung, 2005). However, while students questioned
the usefulness of an academic paper and a research-focus in a competitive business world, notsurprisingly given its academic-focus, the FYP was seen as useful for transition into graduate
school and postgraduate studies, but not as a transition into the workforce. Some representative
comments on the utility of the FYP in terms of student’s expectation in the final year study
include:-‘After doing this FYP, I realize I don’t want to stay in the academic field’ (Humanitiesstudent).
-‘I wonder how an academic thesis is considered to be helpful in a businessworld’(Humanities student).
-‘FYP is not useful to me, because I am not staying in the academic world’ (Humanitiesstudent).
-‘My FYP cannot support my transition to workplace, it is too concentrated and I cannot

learn anything from other fields.’ (Science student)-‘FYP is unhelpful in my transition, because I think I probably won’t stay in the same

field anymore’ (Engineering student).
Highlighting the limited practical utility of their FYP, students reported their experience as
supporting successful job application, but not workplace transition. As one student noted:-‘I expect my degree is to be used for job application, but my research type FYP cannot

assist transition [into the workplace]. If my FYP was to develop a software application [forsmart phone], I think it would help my transition’ (Engineering student).

	
  

137

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Although the majority of students viewed their FYP as unsuitable in terms of transition
characteristics, some students acknowledged a transition component to their FYP because it is
not purely academic research work.
-‘The compulsory placement at a primary school [allowed me the opportunity to]demonstrate my competence as a social worker’ (Humanities student).
The above findings reveal a tendency for a rigid approach to the FYP; the issue calls for better
curriculum design where the experience supports students’ workplace-based competence. The
responses also provide support for the design approach by UCLA that allows for varied study
options, based on capabilities and interests – a student-oriented capstone activity.

Developing graduate capabilities. The core objective of a capstone activity is in essence toequip students with graduate capabilities suitable for their future development. Reflecting on
the broad impact of the four design characteristics, students commonly reported presentation
skills as having improved because presentation is a compulsory aspects of most FYPs.
-‘I think my presentation skills [have] become better because there are some chances topractice throughout the FYP’ (Engineering student).
-‘You won’t write on your job application form claiming that you have completed a FYP,

unless the organization you are applying for is also concerned about your FYP topic, if not
FYP is just a 6-credit course without further implication’ (Humanities student).
While students reported their presentation skills were improved as a result of the compulsory
assessment requirement, in general students doubted the significance of completing the FYP asa way to demonstrate their abilities to the prospective employers. One aspect that appears lessappreciated is the development of written communications; perhaps a capability that does notappear as immediately relevant, but this is still a key graduate capability (Kember & Leung,2005). Overall, Table 1 is a summary of developmental themes as raised by student voice. As

	
  

138

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

the table shows, integration and transition are reported variable, depending on the focus of theFYP, academic or workplace transition. There is in contrast a general appreciation of thedevelopment of graduate capabilities, most notably presentation skills and the capacity to
conduct an investigative study.

Summary of analysis
The findings suggest that the current approach to the FYP appears to address two design
characteristics, closure and reflection, quite adequately. However, the FYP does not appear toaddress integration and transition. Based on the data, all final year students identify the fourcharacteristics as necessary in a capstone experience, but all students emphasized transition as
most important. In contrast to the emphasis on transition, the overt focus of the FYP is notacademic, with most departments requiring a thesis or a research-like project that allows for avaried approach, nor does it relate to workplace matters. Students concerned about seeking
work upon graduation, do not value the undoubted development of research skills and discipline
knowledge from the FYP. What students appear to want most are chances to apply theirknowledge and improve their soft skills. The desired type of activities could be broadly
described as needing to be authentic and practical in nature. However, for some students, thoseaiming for further studies, the FYP helps realize the general goal of undertaking real academic
research. However, this group was a minority in the final year student body and the FYP as
currently conceived can only be seen as a limited capstone experience, and one without the four
design characteristics.

From the findings, it appears that curriculum design affects successful integration ofknowledge. Some students thought that their previous studies were practical enough andsufficient to help them through the process of the FYP, while others thought the curriculum was

	
  

139

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013not coherent enough and that they were unable to apply what they have learnt in the previous
years. However, the academic focus of a FYP facilitated a smooth transition for the
postgraduate students. Table 2 below outlines a design framework that identifies development
aspects, learning activities and assessment strategies for a capstone experience.
Table 2: Design framework to support a capstone activity
Integration** (I)
Reflection (R)Closure (C)Integrate knowledge
and skills

Reflect on development- academically, sociallyand personally.

Choose the activities and assessment
What are we
How (Authentic
learning activities)
developing/ or(Indicative)assessing(Indicative)
University community
Project work, service
Discipline knowledge

Academic skills, selfdirected learning
e.g. research,collaboration

Self awarenessLeadership skills/
Teamwork/
Interpersonal skills
Problem-solving skills

Citizenship

Project work, grouppresentation, simulation,
thesisGroup presentation/caseanalysis/ Simulation/ Lab
experiment report

Reflection journal /BlogGroup project, teambased activities, Careerplanning
Case analysis/ Group/
Research project/
Simulation
Service learning

Transition (T)Close undergraduate
student life

Transition fromundergraduate studies tobeing self-autonomous
learners.

Authentic assessment
methods(Indicative)

Design Characteristics
(I)
(R)
(C)
(T)

Group project, selfreflection
Presentation, peer
assessment, groupprojects, reflections
Presentation,
participation, peer
assessment

✓

✓

✓

✓

✓

✓

✓

✓

✓

✓

Reflection; Pass/Fail✓✓Group project, peer
assessment, reflection,
presentation
Group project, peer
assessment, reflection

✓

✓

✓

✓

Not assessed

✓

✓

✓

✓

✓

The suggested learning activities in Table 2 are similar to the preferred learning activities extracted
from the findings of the student voice. The most welcomed learning activities were group work,
presentations and fieldwork. These activities were important in the development of interpersonal
skills, communication skills, presentation skills and the practical skills that nearly all employers
are looking for. For the majority aiming to start a career right on graduation, an internship to gain a

	
  

140

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

real working experience is most valued. Generally, these students also wanted a smaller class size,as this enabled richer classroom interactions. Alternatively, students who planned to pursue further
studies wanted more seminars and more teaching assistants available to offer them support andstudy assistance during their final year.

	
  
141

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Conclusion
If higher education is aiming at producing multifaceted graduates who are both confident andcompetent in the future workplace, it seems that the current final year project (FYP) is notadequate in preparing the way. This study reveals the research-oriented FYP is the sole option for
final year students and this activity is limited, failing to provide an integrated experience that isable to satisfy the expectations of the majority concerned with finding employment. The current
FYP is, based on student feedback, only able to meet the needs for research students by facilitating
their transition into graduate study. The capstone design framework illustrate stages that can help
programme designers match student capabilities and learning experience to better assist
knowledge integration and successful transition. Students with higher academic capabilities may
be interested in and capable of handling a demanding research thesis. This would most likely suit
those students more interested in staying in the academic field and continuing to graduate school.
For students less interested in academic studies, the opportunity to choose projects, internships orgroup-based projects will help them attain better transition.

In summary, this study illustrates the idea that students should not be limited to knowing things,
but should also be given a chance to reflect on knowledge and to apply what they have learnt intheir studies for their future workplace. The capstone design dimension of transition captures thisconcept. The limitation of this research is that it was conducted only in CUHK; the study may
however reveal what is happening in a wider education sector. The essentiality of the four
components in a capstone experience conveys an important message the student voice appears to
be saying: that integration and transition aspects are not being emphasized under the current 3year curriculum. In other words, students are stuck within the academic world, and the chances towiden their horizons are thus being limited. For the 4-year curriculum, careful consideration
should be taken in design of the new curriculum that culminates theoretically with a capstone
experience. There is a deep yearning to improve students’ capabilities towards realizing their life	  142

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

long goals. There are also some yet unrealized pitfalls from an academics viewpoint, such as
finding suitable topics, marking thesis and providing feedback. These deserve further inquiry.

	
  
143

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

References

Atchison, P. (1993). Creating the capstone connection. Teaching Sociology, 21 (3), 226-228.
Bailey, J., Oliver, D. &Townsend, K. (2007). Transition to practitioner: redesigning a third year
course for undergraduate business students. Journal of Management and Organisation, 13(1), 65-80.
Brewerton, P. & Millward, L. (2006). Organizational Research Methods. Thousand Oaks, CA:
Sage Publications.
Brooks, R., Benton-Kupper, J. & Slayton, D. (2004). Curricular aims: assessment of a university
capstone course. The Journal of General Education, 53 (3-4), 275-287.
Chickering, A.W. & Schlossberg, N.K. (1998). Moving on: Seniors as people in transition. In J.N.
Gardner & G. Van der Veer (Eds.), The Senior Year Experience; Facilitating Integration,
Reflection, Closure, and Transition (pp. 37-50). San Francisco, CA: Jossey-Bass.
Copenhaver, K. (Ed.). (2011). Capstone Majors. In UCLA General Catalog 2010-11. Retrieved
Aug. 20, 2011 from: http://www.registrar.ucla.edu/archive/catalog/201011/uclageneralcatalog10-11.pdf.
Cuseo, J.B. (1998). Objectives and Benefits of Senior Year Programmes. In In J.N. Gardner & G.
Van der Veer (Eds.), The Senior Year Experience; Facilitating Integration, Reflection,
Closure, and Transition (pp. 21-36). San Francisco, CA: Jossey-Bass.
Durel, R.P. (1993). The Capstone Course: A Rite of Passage. Teaching Sociology, 21 (3), 223-225.
Education Commission. (2000). Reform Proposals for the Education System in Hong Kong.
Retrieved Feb. 16, 2011 from: http://www.e-c.edu.hk/eng/reform/annex/Edu-reformeng.pdf
Hauhart, R.C. & Grahe, J.E. (2010). The Undergraduate Capstone Course in the Social Sciences:
Result from a Regional Survey. Teaching Sociology, 38 (1), 4-17.

	
  
144

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Heinemann, R.L. (1997). The Senior Capstone, Dome or Spire? Paper presented at the 83rd NCA
Annual Convention, Chicago, IL.
Henscheid, J.M. (2000). Professing the Disciplines: An Analysis of Senior Seminars and Capstone
Courses (Monograph N. 30). Columbia, SC: University of South Carolina, National
Resource Center for The First-Year Experience and Student s in Transition.
Henscheid, J.M. (2008). Institutional efforts to move seniors through and beyond college. New
directions for higher education, 144, 79–87. doi: 10.1002/he.328
Henscheid, J.M., & Barnicoat, L.R. (2001). Senior capstone courses in higher education. In
Encyclopedia of education (2nd ed.). New York: Macmillan Reference.
Heppner, P. P., & Heppner, M. J. (2004). A guide to successful theses, dissertations, andpublishing research. Pacific Grove, CA: Brooks/Cole.
Holdsworth, A., Watty, K., & Davies, M. (2009). Developing Capstone Experiences.
Melbourne: The University of Melbourne: Centre for Study of Higher Education.
Huber, M.T., & Hutchings, P. (2004). Integrative learning: mapping the terrain.
Washington, DC: Association of American Colleges and Universities
Jervis, K.J., & Hartley, C. A. (2005). Learning to design and teach an accounting capstone. Issues
in Accounting Education, 20(4), 311–339.
Kember, D., & Leung, D.Y.P. (2005). The influence of active learning experiences on thedevelopment of graduate capabilities. Studies in Higher Education, 30(2), 155–170.
Kerka, S. (2001). Capstone experiences in career and technical education. Practice application
brief, 16, Retrieved September 10, 2010, from http://eric.ed.gov/PDFS/ED456333.pdf
Kift, S., Field, R., & Wells, I. (2008). Promoting Sustainable Professional Futures for Law
Graduates through Curriculum Renewal in Legal Education: A Final Year Experience(FYE2). eLaw Journal, 15(2), 145–158.
Rosenberry, J. & Vicker, L.A. (2006). Capstone courses in mass communication programs.
Journalism & Mass Communication Educator, 61(3), 267–283.
	
  
145

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Rowles, C.J., Koch, D.C., Hundley, S.P., & Hamilton, S.J. (2004). Toward a Model for capstone
experiences: mountaintops, magnets, and mandates. Assessment Update, 16(1), 1–2, & 13–
15.
Schroetter, S.A., & Wendler, M.C. (2008). Capstone experience: analysis of an educational
concept for nursing. Journal of Professional Nursing, 24(2), 71–79.
doi:10.1016/j.profnurs.2007.06.019.
Schubert, F.R. (2009). Capstone experiences to support school research events. In Bullet (No. 27).
Retrieved 20 January, 2010, from:http://www.bioscience.heacademy.ac.uk/ftp/newsletters/bulletin27.pdf.
The Chinese University of Hong Kong. (2011). New Undergraduate Curriculum from 2012.
Retrieved Feb 19, 2011 from: http://www.cuhk.edu.hk/334/english/majorprogrammes/capstone.html.
The Higher Education Academy. (2006). Employability in higher education: what it is – what it isnot. Learning & Employability. York: Yorke, M.
University Grants Committee. (2010). Aspiration for the Higher Education System in Hong Kong .
Retrieved May 6, 2011 from:http://www.ugc.edu.hk/eng/doc/ugc/publication/report/her2010/her2010-rpt.pdf.
Wood, L. (2007). The transition to professional work. In Gazatte (Vol. 34, No. 5). Retrieved 19
January, 2011 from:https://www.austms.org.au/Publ/Gazette/2007/Nov07/Gazette34(5)WebVersion.pdf#page=
8

	
  
146

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  

	
  
147

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

	
  

148

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Using a Blog to Facilitate Extensive Reading: An Exploratory Study

Magdalene Meow Khee Chew
Catherine Cheng Kiat Lee

	
  

149

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Abstract
Research shows that extensive reading (ER) has many benefits for language
acquisition. The challenge today is making ER appealing to the digital generation. Fora possible solution, it is pertinent to look to the social media embraced by today’s
youths. This study was conducted to explore the use of the blog as a space for sharing
peer-selected reading material to enhance reading motivation among English
Language learners. The research questions that guided this study are whether the blog
is a viable tool to facilitate ER, and how students perceive the use of this Web 2.0 toolfor ER. The sample comprised two groups of students in a private university in
Malaysia: 12 undergraduate students in a remedial language class and 18 students
enrolled in an English for Specific Purposes (ESP) course. A blog was set up for eachgroup of participants to post reading materials of their choice for blog members toread over a period of 8 weeks. A questionnaire was administered at the end of thestudy together with focus group interviews. The qualitative approach enabled insights
into process and attitudes. The results showed that the students were positive about
the use of the blog for reading beyond the classroom but required tangible rewards
and complementary activities to reinforce their motivation to participate. Most
significantly, the results of this study indicate that the blog is a viable tool for
facilitating ER.
Keywords: Blog, Extensive Reading (ER), Web 2.0

	
  

150

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Introduction
Few language teachers today would argue about the benefits that Extensive Reading(ER) can have for the language learner. Research over the past decade underscores theusefulness and effectiveness of this reading strategy which is also referred to as‘pleasure reading’ (Mikulecky, 1990 cited in Day and Bamford, 1997), and which
Krashen describes as ‘self-selected voluntary reading’ and ‘recreational reading’(2006, p.2-3).

Extensive reading has been found to improve reading skills,

vocabulary, spelling and writing (Krashen, 1993). In addition, ER strengthens thestudent’s motivation to read more (Hayashi, 1999).

With such an impressive ‘resume’, ER should be employed in every English
Language programme. Yet, the reality is that it is not. The reluctance of teachers to
implement an ER programme in their classrooms has been attributed to a host ofdeterrent factors amongst which are cost, time constraints, and the work required to
set up such a programme (Day & Bamford, 1998:46). A decade later, problems still
exist. Macalister’s (2010) study on the attitudes of teachers of University preparation
courses in New Zealand towards ER revealed that time constraints within teaching
programmes and uncertainty about university students’ view of the relevance of areading programme discouraged them from implementing ER in their institution.
Another study on Taiwanese language educators highlighted logistical issues
concerning the management of the process of ER (Sun, 2003). The online reading
programme designed by Yu-Chih Sun (2003) to overcome the perceived problems of
implementing ER among EFL learners is based on a structured system that fulfills therequirements for ER. However, it was reported that the rigid system had problems
such as workload issues, inflexibility and students’ aversion for writing reflection.

	
  

151

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

We posit that the implementation and effectiveness of an ER programme is not only
hampered by logistical issues but also by the learners’ attitudes. It is an establishedfact that the young people sitting in the classrooms today have grown up withtechnology and are called by many names including digital natives (Prensky, 2001),
Millennial students (Elam, Straatton & Gibson, 2007 in Lomicka & Lord, 2009) andthe Net Generation (Tapscott, 1998, 2009). This group is profiled by Prensky (2001)
as needing information fast, preferring random access and thriving when networked.
These learners’ preference for an experiential, interactive and image-rich environment(Tapscott, 1998) also indicates a gap between their text consumption habit outside theclassroom and the traditional textual experiences in the classroom. Educators have to
respond appropriately to the fact that the typical teenager today encounters texts thatbear little resemblance to the book on a daily basis (Bigum & Lankshear, 1997,
Healy, 2000, in Green & Campbell, 2003). To increase the potential of success of the
ER programme, it is only logical to look to digital technology and media thisgeneration is comfortable with for an alternative platform to the traditional approach.
This study examined the use of the blog to facilitate an ER programme and is basedon the rationale that the characteristics of this Web 2.0 tool can support the principlesof this reading programme while avoiding some of the inhibitive problems.

Characteristics of the Weblog
The Weblog or blog is a second-generation asynchronous Internet tool that is defined
as an online hypertext journal that others read and react to (Blake, 2008). Its primary
purpose is to provide a space for the blogger to share writings with the online
community who then respond in writing. The interface is easy to negotiate and allows
pictures as well as audio and video clips to be included. The postings are displayed in

	
  

152

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013chronological order and automatically archived so that a visitor can search for andread earlier entries (Sharma & Barrett, 2007).

Its versatility is underscored by

Bhattacharya and Chauhan (2010) who describe this popular Internet communication
tool as a ‘dynamic place that is connected by time and topic and a frequently posted
list of interesting websites, or a personal diary of events and thoughts, or a
combination’. Their research on learner autonomy through blogging yielded positive
results.

The blog is not a new technological platform but Lomicka and Lord (2009) rightly
point out that Web 2.0 “is really an attitude and not a technology” and is about
existing technologies being harnessed to do more and different things, a notion that
supports this exploration of the blog for enhancing reading rather than writing which
would be closer to its original purpose. There are sufficient reasons to support the ideaof using the blog as a repository for authentic target-language resources easilysourced from the Internet. Setting up a blog for a class of students to share readingmaterials requires minimal time and cost. The blog can be accessed anywhere using
any device that allows connection to the Internet such as laptops, tablet computers andmobile phones. The collaboration made possible through an easy to access platform
such as a social networking tool like the weblog could create a supportive social
climate that helps motivate the learner to participate in ER.

The blog and the key elements of ER
Day and Bamford (1998, 2002) offer 10 principles for ER which deals with thecharacteristics of ER and the conditions and methodology needed for its success. How
the blog is able to support each of these principles is explained below:

	
  

153

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

1. The reading material must be easy
The reading material would be selected by the students for the students. It is logical toassume that the chosen reading materials would be filtered and deemed accessible tothe target audience. This implies that the materials posted in the blog would becomprehensible to the members.

2. A variety of reading material on a wide range of topics must be available.
The blog members would have the Internet as a resource for reading material. There isno question about the Internet being a rich resource of a vast variety of readingmaterial. According to a Web server survey conducted by Internet Services company
Netcraft in August 2011, there are more than 460 million websites in existence. Not
only do these sites supply text on every topic imaginable but they also provide a wide
variety of text types including comics, jokes and video clips which are the staple fareof the visual-hungry digital natives (Prensky, 2001).

3. Learners choose what they want to read.
The blog would serve as a repository of reading materials deemed interesting andsuitable for reading by the learners themselves. The process requires participants tofirst choose from what they enjoyed reading to share on the blog, and then to selectfrom the shared materials what they want to read. Freedom of choice is exercised at
two points of the process. Not only would they choose what they want to read but
they would also recommend what they enjoy reading to their friends. Hence, thematerials are peer-selected and self-selected.

	
  

154

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

4. Learners read as much as possible.
The task of sharing reading material on the blog would ensure that students read fortwo purposes. First, the students have to search for reading material to share. In otherwords, they have to read the material to determine its suitability for sharing. Next,
they are instructed to read what other members have posted which provides for more
reading. This time, it is a more ‘conscious’ reading exercise.

5. The purpose of reading is usually related to pleasure, information, and general
understanding.
The Internet allows for easy access to a wide variety of reading materials. With thefreedom of choice that the participants are given, they would naturally look for what
interests them and gives them pleasure.

6. Reading is its own reward.
The blog is for students to read and share what they read so that they can read evenmore. There are no structured activities or comprehension and language exercises tointerfere with the reading activity itself. It is believed that learners will derive bothpleasure and language benefits from reading the blog posts.

7. Reading speed is usually faster rather than slower.
If the reading material is interesting and easy to understand, the reader will readfaster. Again, this is made possible by virtue of the resource being the Internet and the‘chooser’ being a peer who likely has similar interests and language ability.

	
  

155

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

8. Reading is individual and silent.
Participating in this blogging activity requires the students to read outside theclassroom at their own pace and leisure. When to read and how much to read is leftentirely to the individual. Although the students are instructed to post at least once aweek and to read at least half of what is posted as a guide, they are under no duress as
there is no strict monitoring or accounting involved.

9. Teachers orient and guide their students
This study focused on the use of a Web 2.0 tool in ER and sought to examine its
efficacy in motivating students to read. Orientation and guidance were provided at
the initial stage of the experiment. Thereafter, the instructor kept in the background,surfacing when necessary to give verbal encouragement to the class to continue
participating.

10. The teacher as a role model of a reader.
For ER to be effective, the teacher is expected to blaze the trail for the students. Witha reading blog, the instructor can easily access what the students are reading and read
what they read. The students will see the instructor’s comments on the posts which
can be an incentive to them to read as well.

Theoretically, the blog could help eliminate the deterrent factors that make language
practitioners shy away from ER. It is then necessary to examine it from the practicalperspective. Is the blog viable as a platform for an ER programme? Can it support thecharacteristics of ER?

	
  

156

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

The other dimension that needs to be assessed in relation to the practicability of using
this online social network tool is the users’ perception and attitude. The learners’
receptiveness towards using what they regard as a social medium for academic
purposes is important to the successful implementation of the programme.

Methodology
This study explored the use of the blog as a space for language learners to sharereading materials for the purpose of Extensive Reading. The primary focus of thestudy is to examine the viability of the blog in facilitating ER and to gain insights into
the perception of the participants towards the use of this social network platform for
ER. This paper employs the qualitative approach with a detailed explanation of thefindings from the questionnaire administered and the focus group interviews
conducted.

The Participants
The study involved two groups of students at a private university in Malaysia. The
groups were taught by one of the researchers in two separate semesters. Twelve
students in a remedial English language programme made up the first group. They
were a multinational group made up of students from Malaysia, Indonesia, Myanmar
and China, with Indonesians having the largest representation (7). These first year
degree students majoring in information technology and computer science wereenrolled in the language programme to raise their proficiency from pre-intermediate
to intermediate level. The programme was an intensive 12-hour per week course that
focused on language skills. The second group consisted of 18 students in a foundation

	
  

157

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

programme. They consisted of Malaysians with those of Chinese descent making up
two-thirds of the group. The students were taking Technical English, an English forSpecific Purposes (ESP) course which was a core subject meant to equip them withlanguage skills relevant to their specialism. Their English language proficiency
ranged from pre-intermediate to advanced levels.

Procedure
The objective of the planned extensive reading programme was first explained to eachgroup. The blog for the Remedial English class was created by the instructor using
Blogger, a free blog-publishing platform, and given the name CPE Readers Club. The
ESP group created their own blog using WordPress, another popular free blog
publishing tool. The group came up with their own name for the blog – Techreaders.

The participants were instructed to post materials in their respective blogs for theircourse mates to read for a period of eight weeks. The guidelines for participation werealso posted in both the blogs.
GUIDELINES FOR MEMBERS
You can...
...share articles, news, comics, stories, poems, jokes & riddles
You should...
...provide a brief description of what you want to share...post something at least once a week...read at least half of what has been posted...comment on what you have read
You must not...
...post material with sensitive content (e.g. about politics or religion)...post pictures without text
Figure 1: Guidelines for members of the blogs (Extracted from: http://cpereaders.blogspot.com/)

	
  

158

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

The students were allowed to select any form of reading material to fulfill the‘freedom of choice’ aspect of ER. Evidence that they had read a post was derived
from their comments. They were allowed to respond very briefly to avoid deterring
them from reading. If they were required to write at length, it could be perceived as
too much work which was the case with the Taiwanese students using the ERO
system (Sun, 2003).

The wide variety of Web content that is easily accessible also necessitated defining
perimeters for this activity. Hence, there was the reminder to avoid materials of asensitive or offensive nature such as politics and religion.

The lecturer planned to stay invisible and remain a passive observer throughout theeight weeks, only giving verbal reminders to the class to keep posting and reading.
This was to minimize the influence of external factors and maintain the study’s focus
on the use of the social media platform.

Feedback and Evaluation
A post-study questionnaire focused on gaining information on the students’participation (frequency in posting and reading), the procedure involved (how theylooked for reading material and how easy or hard it was), their feelings about theactivity of posting and reading, and their opinion of the blog (whether it needed
improvement, how to motivate students to participate and its usefulness, effectivenessand relevance to language learning).

	
  

159

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Seven students who formed the focus group were also interviewed. The focus of theinterview was for a more in-depth examination of the students’ understanding of ER
and its benefits, their views of reading on the Net, the collaboration aspect of thereading blog, and their level of involvement. The students were also asked to voice
their opinions of the blog. These seven represent the spectrum of participants – fromthe active participants to the observer.

Findings

The viability of the blog for facilitating ER
The results of this experiment are presented from the viewpoint of how they support
the ten ER principles (Day & Bamford, 1998, 2002). The approach adopted in thisstudy required the reading material to be selected by the students for the students.
The participants reported that they evaluated what they read for suitability in terms oflevel of difficulty and appeal before sharing it. A focus group member described theprocess involved in selecting reading materials succinctly: “Two steps – browsing andcoming across something worth sharing, ask ‘Is it appropriate to share?’ Yes? Then
post.” This implies that the materials deposited in the blog were comprehensible tothe members and therefore supports the principle that reading material has to be easy.

The participants who reported that it was easy to look for reading material comprised
46 per cent. Of these, half said it was because of the Internet which makes available awide range of reading material (principle number 2). However, there were five
students who found the task difficult of deciding what to share with their friends on
the blog. There were also a couple of students from the Remedial class who said they

	
  

160

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

had difficulty understanding what they read on the Internet. This narrowed down theiroptions for what to share which made this part of the programme difficult.

Freedom of choice was central to the whole experience. The participants were free toselect any reading material to share within the set perimeters. They could decide
whose posts to read, when, where and how to read, and also whether to respond andcomment or not. In this liberated environment it was found that the main criterion
used by 46 per cent of the participants to pick what to post was peer influence.
Another 18 per cent specified that they looked for humorous materials that would
make the others happy. They had a clear sense of the audience when they selected
what would interest their friends rather than what interested them. The outcome was
86 per cent of the participants reported that they enjoyed reading the peer selectedmaterials on the blog. Also significant is the fact that over two-thirds of theparticipants maintained that they enjoyed looking for materials to share with theirfriends. About a third of this group said they gained new information and knowledge
from this exercise. This observation is also relevant to the fifth principle which relates
the nature of the reading materials and the participants’ interests. Clearly, pleasure,
information and general understanding form the basis of the students’ decision
making.

The 2-stage approach which required students to first read to search and post, andthen to read the shared materials increased the learners’ exposure to texts. Forty percent of the students contributed to the content in the blog at least once a week. Sixty
per cent of the members read the blog at least once a week. These participants had theopportunity to read as much as possible which supports the fourth principle of ER.

	
  

161

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

The experiment yielded two significant results that relate to the principle that readingis its own reward. The first is that the participants did enjoy both the reading tosearch stage and reading the blog stage. However, more participants (86%) reported
that they enjoyed reading their friends’ posts at the blog stage than those who liked
reading at the search stage (68%). The second discovery was unexpected. Althoughreading was perceived to be academically rewarding, a number of participants (18 percent) felt that some form of activity such as discussion or language exercise based on
the posted material would have motivated them to be more active on the blog. Even
more surprising was the suggestion by 25 per cent of the participants that some prize
or token be offered as an added incentive to the readers.

Fulfillment of the seventh principle can be inferred from the participants’ affirmative
responses about their enjoyment when reading the posts on the blog. A significant
percentage (86%) reported that they enjoyed reading the shared materials.

The eighth principle states that ER has to be individual and silent. The students readat their own pace during their free time. The largest group (32%) read the blog at least
once a week while the second largest group (29%) read whenever they felt like itwhich varied between twice a week to once in 2 weeks. They maintained that thefrequency of their reading activity depended on whether the materials were interesting
and whether they had free time.

	
  

162

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

The last two principles of ER emphasize the role of the instructor. In this study, thelecturer explained the objectives of ER and introduced the blog as the platform for the
programme. Guidelines and rules of conduct were given and then posted on the blog
as a permanent reference for the blog members thereby meeting the ninth principle.

The researcher deviated from the tenth principle which requires the teacher to be arole model of a reader for the reasons already explained in the previous section of thisreport. In the first part of the study which involved the Remedial Class, the instructor
kept in the background throughout as planned, reading the posts but withholding
comment. This was to find out whether the students were motivated by the social
media platform to keep the momentum they had. It was observed that by the fourthweek, that is, mid-way through the experiment, the posts became less frequent andactivity slowed down. A number of the participants expressed their wish to see thelecturer involved in posting reading materials, and initiating discussion of what wasposted.

With this discovery, the instructor played a more active role in the blog for the ESP
class in the following semester. However, that role was limited to reading andcommenting on what was read. The teacher abstained from posting reading material
so as to maintain the learners’ freedom of choice of reading material. This is based on
the rationale that any material selected by the teacher would be obligatory reading forthe students. It was found that the students preferred the teacher to be even more
active. One student felt that the blog members became less motivated with time andstronger leadership would have restored interest.

	
  

163

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Students’ perception of the blog as a tool for ER
The students viewed the use of the blog for this reading programme favorably.
Feedback on the effects and benefits of using this approach was obtained via the poststudy questionnaire and focus group interview.

Usefulness and Effectiveness in supporting language learning
Three quarters of the participants felt that every English class should have a reading
blog. They specifically said that it would help improve their reading and writing
skills, vocabulary and general language. There were a few detractors (14%), however,
who thought the reading blog was ineffective or uninteresting.

Effect on motivation to read
On whether the blog motivated the students not only to read more, but to read more
carefully and purposefully, the focus group were unanimously affirmative on all
points. All the focus group members thought that it was a more interesting way toread. One said it was fun while another explained that the collection of readingmaterials on the blog allowed her to encounter new kinds of reading materials that she
would otherwise not read. A more comprehensive explanation was offered by one
member of the focus group: “We are the Y Gen. Content flows faster on the Net andso better captures interest. I cannot see people needing books any more.”

One of the reasons for the participants’ liking the reading blog has to do with thedesign of the blog. They liked the versatility of the blog which allowed the design tobe changed, the attractiveness of the design and the user-friendly interface.

	
  

164

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Benefits from collaboration
Of the 86 per cent who enjoyed reading the posts in the blog, three quarters gave
reasons that point to the social aspect of the blog. The responses of ten of the students
reveal a consciousness of their relationship to the other participants. For example,
two participants said they appreciated the hard work put in by their friends to findmaterials to share with them. Eighteen per cent of the participants endorsed the blog
because it let them share their knowledge and also reading materials that interest themwith their friends which is further evidence of the advantage of the collaborative
aspect of this social online platform. One of the students interviewed reinforced thisidea saying that the blog gave them a greater sense of community. Another student
stated that the Internet is so ‘huge’ that it is not easy to look for specific knowledge,
but with the reading blog, others did the searching and posted materials that advanced
his knowledge.

Ideas for improving the reading blog
Although a decisive proportion of the students reported that they approved of thereading blog, it was observed that they did not participate as actively as they should.
Their responses to the questionnaire items about what would have made the blog
more effective plus input from the focus group indicated where the problems were.

A surprising 25 per cent of the students thought that a reward such as a prize, gift or
token would motivate students to participate in the blog.

Another 21 per centsuggested that the design of the blog be improved further. Complementary activities
such as quizzes, contests, brain teasers, discussion and even exercises were thought toencourage more participation. Five participants mentioned that the lecturer needed to

	
  

165

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

be more active and to motivate them. One of the focus group members pointed out
the need for more ‘leadership’ in the ER programme. The lecturer’s participation wascited by several students to be a source of motivation.

In addition the posts

themselves were said to augment reader engagement if they contained humour,pictures, music and videos.

One of the focus group members who was not as enthusiastic about the reading blog
as his counterparts suggested that linking the blog to Facebook would generate moreinterest and attract more participation. Another focus group member echoed thissentiment with the rationale that Facebook is so popular with these young people thatlinking this social network to the blog would help reduce the academic-ness of theactivity and remove the pressure of perceived expectations from both peers andinstructor, thus making this approach to ER more casual and engaging.

	
  

166

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

Conclusion
This exploratory study confirms that the blog is a useful and viable platform forsupporting an ER programme. Its benefits are derived from allowing easy access to avast resource of reading materials to cater to the diversified interests of a group oflearners and the convenience and ease with which a group can select reading materials
and deposit them in a common space for members to exercise the free will to read, notto read or to read again.

The students’ favorable perception towards this social networking tool is anotherimportant endorsement for this alternative approach to ER. This study confirms thatthe use of this Web 2.0 tool appeals to this representative group of the Net Generation
because it permits them to collaborate with their peers and to share their interests inthe form of text. In the words of one of the participants, this generation “crave[s] theirfellowmen’s approval more than anything else” and they “by nature like to share andknow about each other”.

While this Web 2.0 platform appears to have passed the test as a tool as an alternative
approach to a traditional language learning enhancement programme, there are someconcerns that need appropriate response and further examination. One such issue isthe difficulty faced by the learner in selecting reading materials.

In spite of, or perhaps it is more accurate to say because of, the huge resource of
online reading materials, searching for reading materials to share was not as enjoyable
as reading what was shared especially among the lower proficiency group. Hence,
having too many options is also a deterrent. A possible solution is to provide a list of

	
  

167

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

recommended websites for those learners who need assistance at this stage of the
programme while still allowing the rest who are more adept at Web searching to go
beyond that list.

Contrary to Bhattacharya and Chauhan’s (2010) research findings, the results of thisstudy have debunked the idea that this interactive online platform could make thelearner more independent and responsible in their learning. The warning by Day andBamford (1998, p.126) concerning the lack of encouragement and concern on the partof the instructor would definitely lead to diminished interest and participation in an
ER programme was found to be valid. Evidently, the use of the Web 2.0 resource
does not eliminate the need for the teacher or lecturer’s catalytic role. In spite of theaccessibility, convenience and appeal of this online platform that encourages
autonomous learning, the students were still unable to abandon their need or
preference for traditional hand-holding. The use of the blog for ER was found to haveminimum impact on learner autonomy which is evident in the students’ preference formore active participation by the lecturer, structured activities to accompany thereading and some form of reward for their participation.

Further investigation is recommended especially on how and whether using theweblog and other social networking sites can significantly increase the effectivenessof ER in improving language. Once it can be established that the online platform cansupport the ER programme in all aspects of language learning, it could provide
language instructors with an alternative approach that would help remove thereservations currently impeding the implementation of this worthwhile programme.

	
  

168

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

References
Bhattacharya, A. & Chauhan, K. (2010). Augmenting learner autonomy throughblogging. ELT Journal Volume 64/October 4, 2010. Retrieved March 5, 2011
from: http://eltj.oxfordjournals.org/content/64/4/376.full.pdf+html.
Blake, R.J. (2008). Brave New Digital Classroom: Technology and Foreign
Language Learning. Washington D.C.: Georgetown University Press.
Day, R.R., & Bamford, J. (1998). Extensive Reading in the Second Language
Classroom. U.K.: Cambridge University Press.
Day, R.R. & Bamford, J. (1997). ‘Extensive Reading: What Is It? Why Bother?’ The
Language Teacher Online. Retrieved March 5, 2011 from:http://jalt-publications.org/old_tlt/files/97/may/extensive.html.
Day, R.R., & Bamford, J. (2002). Top Ten Principles for Teaching Extensive
Reading. Reading in a Foreign Language. 14(2), 136-141.
Green, D. & Campbell, R. (2003). (Eds). Literacies and Learners: Current
Perspectives. NSW: Prentice Hall.
Hayashi, K. (1999). Reading strategies and extensive reading in EFL classes. RELC
Journal, 30, 2, 114-132. . Retrieved March 5, 2011 from:http://news.netcraft.com/archives/2011/
Krashen, S. (1993). The Power of Reading. Englewood, CO: Libraries Unlimited.
Krashen, S. (2006). Pleasure Reading Young Learners SIG Spring Issue (April), p.65.
Lomicka, L. & Lord, G. (2009). (Eds.) The Next Generation: Social Networking andOnline Collaboration in Foreign Language Learning. USA: Calico
Monograph Series Volume 8.
Macalister, J. (2010). Investigating Teacher Attitudes to Extensive Reading Practices
in Higher Education: Why Isn’t Everyone Doing It? RELC Journal April 2010

	
  

169

The IAFOR Journal of Education

Volume 1 - Issue 1 - Spring 2013

41: 59-75. Retrieved March 5, 2011 from:http://rel.sagepub.com/content/41/1/59.full.pdf.
Prensky, M. (2001). “Digital Natives, Digital Immigrants” On the Horizon, vol 9, no.
5 (October 2001) pp. 15-24. Retrieved March 2, 2011, from:http://www.marcprensky.com/writing.
Sharma, P., & Barrett, B. (2007). Blended Learning: Using technology in and beyond
the language classroom. U.K.: Macmillan Education.
Sun, Y.C. (2003). Extensive reading online: an overview and evaluation . Journal ofComputer Assisted Learning, 19, 438-446.
Tapscott, D. (1998). Growing up digital: The rise of the net generation. New York:
McGraw-Hill.
Tapscott, D. (2009). Grown up digital: How the net generation is changing yourworld. New York: McGraw-Hill.
	
  

	
  

170