XCourEva: XML-Based Course Evaluation System
Somchai CHATVICHIENCHAI

Abstract
This paper describes XCourEva, a new Web-based course evaluation system. XCourEva
is intended to assist administrative staff in defining a default questionnaire for eachacademic unit as well as additional customized questionnaires for individual instructors.
The advantages include easy development and maintenance of the default and additionalquestionnaires, automatic generation of Web pages for filling out a questionnaire using
XML, and bi-directional communication between the instructor and anonymous students.
Analysis of course evaluation results from various points of view is aided byautomatically generated summaries, and a Microsoft Excel! spreadsheet is generated that
presents a consolidate view of students’ feedback on the courseware. Instructors canharness the vast data presentation capabilities of the Microsoft Excel! 2003 package toreplace the bar chart with any convenient type of graph and to compare their evaluation
results with those of previous academic years.
Keywords: XML; analysis tool; Web; evaluation; anonymous.

1. Introduction
Many colleges and universities conduct course evaluation programs to help instructors improveon future course offerings and teaching methodologies [1, 2]. The University of Nagasaki beganthese endeavours in 2003, which are conducted using machine-readable questionnaires during thelast week of class in each semester. Over this period, however, three main problems have been
identified with the rather old-fashioned paper-based method. The first concerns the high overhead
costs involved in both conducting the evaluation and updating the questionnaire yearly. This cost
includes, inter alia, that of printing newly designed questionnaires, processing the completedquestionnaires, and preparing reports for each instructor. The second problem concerns the factthat the single questionnaire developed cannot suitably cover the requirements of all academicunits. Some instructors report that the questionnaires do not include specific questions that theywant to ask and so resort to their own independent evaluations. The last problem is that because
evaluation reports are only available four months after the end of each semester, feedback cannot
be immediately incorporated in a dynamic manner. A new Web-based course evaluation systemcalled XCourEva is proposed to address these drawbacks. The objective of this work is todescribe how XCourEva is being developed to meet the specific needs of the university’s
academic units, in particular, (1) the ability to customize questionnaires to individual academic―２
３―

Journal of the Faculty of Global Communication, University of Nagasaki No．
１１（２０１０）

units and instructors, (2) the need to improve the quality of students’ feedback, (3) the ability togenerate proper statistical data, and (4) a low system development and maintenance cost.
Related workThe assessment of the quality of teaching in university courses has always been an issue ofgreat interest in the academic domain and software development industry. The Free Assessment
Summary Tool (FAST !) [3], developed at the Mount Royal University, Canada, together with anindependent software designer, provides instructors with a user friendly tool for developing Webbased surveys. Survey results can be automatically tabulated and made available on the Web tothose with proper authorization. CoursEval TM [4] is a commercial Web-based tool that allows for
self-and-peer evaluation, which can be an effective tool for team-oriented courses. The peer
review function, in particular, provides a means for students to evaluate other students within aclass over the term of the course. SW3A [5] is a Web-based tool under development to support theassessment of academic activities and allow for students’ follow-up. With regard to the latter, SW3
A will allow course councils to monitor the development of teaching plans in order to guarantee
precise syllabus fulfillment. The proposed system will also allow publishing of data related to theteaching plans (syllabi), grades obtained in the applied evaluations, and students’ attendance foreach of the offered disciplines. IU-EVAL [6] is a Web-based course evaluation system developed
at Indiana University (IU) South Bend. The design goal was to develop a customizable andscalable electronic evaluation system that fulfills the unique needs of an academic institution.
Furthermore, the researchers sought to make the system freely available to the larger academic
community through the World Wide Web. Note, however, that none of these tools provide afacility for bi-directional communication between an instructor and anonymous students.

2. Issues in Developing XCourEva
2.1 Customization of questionnaires for individual academic units and instructors
The ideal solution is a system that enables individual academic units to set their own
questionnaires. They should be able to define different default questionnaires for different course
types (e.g., faculty-level general lectures and department-level specialized labs). The default
questionnaire should be used for all the courses in the academic unit and cannot be edited byindividual instructors. This function is realized by having individual academic units send theirdefault questionnaires to the educational affairs office for entry into XCourEva. Furthermore,individual instructors can submit additional questions that pertain directly to their courses througha user interface provided by XCourEva. Therefore, the questionnaire template for a course
consists of the default questionnaire of the academic unit and additional questions sent by theinstructor for the course. All questions are answered on a scale of 1−5, where 1 denotes strong
disagreement, 5 denotes strong agreement, and 3 denotes neutral. Feedback on the additionalquestions is sent only to the course instructor.
2.2 Improvement in the quality of student feedback
The ideal solution is a system that enables students complete their evaluation anywhere and―２４―

Somchai CHATVICHIENCHAI：XCourEva: XML-Based Course Evaluation System

anytime, i.e., where and when they have access to a computer with a connection to the university
network. XCourEva realizes this by exploiting Web technologies such as HTML, XML [7], and
ASP [8]. In addition, XCourEva provides the ability for instructors to conduct course evaluation
as often required during a semester so that they can dynamically collect and incorporate student
feedback to improve their teaching before a semester finishes. They can modify the additionalquestions that they use in the default questionnaire each time they conduct an evaluation. Andeach questionnaire includes a section on suggested improvements, where students can, in fullconfidentiality, input information they believe will be useful to the instructor. XCourEva also
allows for real-time online chat between the instructor and anonymous students by which theinstructor can explain how feedback will be acted upon.
2.3 Flexibility in statistical analysis of data
The Web-based system stores feedback data in XML format and summarizes the results on
the basis of course code. The system then coverts this data into a Microsoft Excel! (hereinafter,Excel) file [9] that instructors and department heads can use for analysis from various points ofview (see section 3). Instructors can download an Excel file containing a statistical summary on aselected course. The vast functionality of Excel allows them to create an array of graphs andcompare data with those from previous academic years.
2.4 Maintaining student anonymity and data confidentiality
The goal is to define a proper access control policy for feedback and statistical data. In
XCourEva, users are classified as students, instructors, and staff. Students are allowed to fill out
questionnaires pertaining to courses that that have enrolled for. Instructors are allowed to enter
additional questions pertaining to their courses and are only allowed access to statisticalsummaries on these courses. Importantly, these data do not include any information that can beused to identify a particular student. Staff are allowed to enter the default questionnaires foracademic units and summarize the statistical data.

3. System Architecture
As shown in Fig. 1, XCourEva consists of the following five programs: (1) a default
questionnaire entry program, (2) additional questionnaire entry program, (3) evaluation
submission program, (4) evaluation result summary program, and (5) evaluation result enquiryprogram.
3.1 Default questionnaire entry program
Authorized administrative staff input the default questionnaires received from the heads ofthe academic units, which XCourEva stores in an XML file. The advantage of using the XML
format instead of a relational database is the reduction in system development cost. Briefly, XMLprovides a way to describe structured data. That is, an XML file is a plain-text file that uses XML
tags to define the logical structure of the document in a hierarchical fashion. XML uses a set oftags to delineate data elements, each of which comprise subelements and attributes. These―２５―

Journal of the Faculty of Global Communication, University of Nagasaki No．
１１（２０１０）

Fig. 1 Architecture of XCourEva system(b)(a)Fig. 2 (a) An example of questionnaire XML file; (b) the schema diagram of questionnaire XML file.

subelements, in turn, contain other subelements, and so on forming the hierarchy. Figure 2(a)illustrates an example of a questionnaire XML file for an academic unit whose code is “0202.”
Here, the first two digits of the academic code are the faculty code, and the last two digits are thedepartment code.
As seen in the schema diagram in Fig. 2(b), a default questionnaire XML file consists ofacademic data (defined by academic element) and questionnaire data (defined by q_sectionelement). A subject_category value “1” denotes a lecture whereas a value of “2” denotes a―２６―Somchai CHATVICHIENCHAI：XCourEva: XML-Based Course Evaluation Systemlaboratory class. Default questionnaires are described by the q_section element, where the valueof type attribute is set to “core.” The content of a questionnaire is described by the text
subelement of the q element. The “core” questionnaires will be used for all the courses in theacademic unit and cannot be edited by the course instructor. As the cardinality of the q element is
1..∞, then the number of questionnaires is greater than or equal to one. The value of the seq
attribute of the q element denotes the sequence number of the questionnaire. The content of thefirst questionnaire is “Overall, this is an excellent course.” In the schema diagram of Fig. 2(b),the comment element is optional. A comment value of “yes” for the second q element informs thesystem that a comment box must be presented. Therefore, the second additional question will beanswered by the students on the 1−5 scale, together with a descriptive comment.
3.2 Additional questionnaire entry programThis program enables individual instructors to enter additional questions that pertain directly
to their courses. The additional questions are defined by setting the value of the type attribute of
the q element to “additional.” Note again that an instructor is allowed to read the default
questionnaire but is not allowed to modify them.
3.3 Evaluation submission program
A student is allowed to participate in course evaluation via computers connected to theuniversity network. After passing authentication, the student selects the course to evaluate from alist box showing the courses that the student is allowed to evaluate. The evaluation submission
program reads the default and additional questionnaire XML file for the selected course andautomatically generates a Web form. Once the students have submitted their entries, furtherchanges will no longer be possible. The program then outputs the results for a subject answeredby a student into an evaluation result XML file.
3.4 Evaluation result summary program
After course evaluation is complete, authorized administrative staff will execute theevaluation result summary program. For each course, this program computes the number ofevaluators and average scores for each question from the submitted questionnaire XML files, andthen generate a course evaluation result XML file that contains the average scores and comments,
as shown in Fig. 3(a). The program also converts each a course evaluation result XML file into anExcel file with a bar chart. In order to reduce programmer burden, the built-in XML data importfunction of Excel is invoked. Briefly, the system manager adds the evaluation summary XML
schema [10] shown in Fig. 3(b) to a workbook called the course evaluation template file throughthe XML map provided by Excel and binds data from elements of the XML schema to cells andcolumns on the worksheet. Thus, the XML data import method automatically imports the datafrom an evaluation summary XML file into the course evaluation template file, which issubsequently saved as an Excel file containing evaluation summary result of the course, as shownin Fig. 4. Note that in this figure, the right window is the XML structure task pane showing theXML schema elements being bound to cells and columns on the worksheet.

―２７―

Journal of the Faculty of Global Communication, University of Nagasaki No．
１１（２０１０）(a)(b)
Fig. 3 (a) example of an evaluation summary XML file; (b) schema diagram of the evaluation
summary XML file

3.5 Course evaluation result enquiry program
The program allows only authorized users to retrieve the course evaluation results. After theuser selects a course from a list of courses that the user is allowed to retrieve, the program opens
the Excel file that contains the selected course’s evaluation results and presents a bar-chart. An
example of an outputted course evaluation result Excel file is shown in Fig. 4. By selecting “yes”
in the comment column, users can download the text file containing the students’ comments on
the corresponding question.

Fig. 4 Screenshot of questionnaire summary result of a course―２８―Somchai CHATVICHIENCHAI：XCourEva: XML-Based Course Evaluation System

Number of completed questionnaires

1,000

2,000

5,000

10,00020,000Time for generating course evaluation
summary XML files (in seconds)

74152496

9011,996(a)(b)Fig. 5 (a) Time taken in generating course evaluation summary XML files; (b) Course
evaluation summary XML file generation test result

4. System Implementation and Performance Evaluation
The programs and data files comprising this system run on the Windows Server 2003 R2(Standard Edition with Service Pack 2) environment. Internet Information Services (IIS) 6.0 [11]is employed as the Web server for questionnaire input, and data retrieval is supported via theMicrosoft Internet Explorer Web browser. XCourEva was developed using ASP.NET technology
in order to efficiently handle dynamically generated Web pages. Since the generation of a course
evaluation summary XML file is the most time consuming task, we use this generation time as aparameter to evaluate system performance. An experiment was performed on a server equippedwith a Pentium! 3.4 GHz processor and 1 GB of main memory, and the results are shown in Fig.
5(a) As seen in Fig. 5(b), a linear relationship was seen between the time taken and the number ofcompleted questionnaires.

5. Conclusion
This paper introduced a new Web-based course evaluation system called XCourEva, which
is being developed to meet the specific needs of individual university academic units. These needs
include the ability to customize questionnaires for individual academic units and instructors, theneed for improving the quality of student feedback, the ability to generate proper statistics, andlow system development and maintenance cost. These needs are fulfilled by using XML
technology: storing data in XML format and converting course evaluation summary XML files
into Excel spreadsheets. Thus, XCourEva offers customized reporting options, exploiting the―２９―Journal of the Faculty of Global Communication, University of Nagasaki No．
１１（２０１０）

multiple data presentation methods provided by Excel. XCourEva also provides the ability forinstructors to conduct course evaluation as often as they want during a semester so that they canobtain student feedback to improve their teaching methodology before a semester is complete.
Each time an evaluation is conducted, additional questions can be added to the questionnaires.

Acknowledgment
The author would like to thank Matsumura Yuta and Kai Yumiko for their hard work anddedication during the analysis, design, implementation, and testing of this system.

References
１．H. Marsh, Students’ evaluations of university teaching, International Journal of Psychology,
n.39, v.5-6, p.40-40, 2004.
２．M.T. Thielsch, Y. Grabbe, K. Haaser and R. Moeck, Course evaluation online: Students’evaluation of university teaching by web based assessment technology, http://www.gor.de/gor05/index_4.html (13 August 2010).
３．J. Paskey, FAST!-Seeing Student Assessment in a Brand New Light, http://www.getfast.ca/fast.pdf (14 Aug 2010).
４．Academic Management Systems, http://www.academicmanagement.com/products/course_eval.php
５．J. Freire and C. José, SW3A: Web-Based System for Academic Assessment and Follow-Up,
Int. Conf. on Engineering and Education 2007, Coimbra, Portugal, Sep 2007.
６．H. Hakimzadeh and L. Williams, IU-EVAL: An Electronic Course Evaluation System,
Proceedings of the 34th annual ACM SIGUCCS fall conference (SIGUCCS’06), pp.131-134,
November 5-8, 2006, Edmonton, Alberta, Canada.
７．W3C. 2006.“Extensible Markup Language (XML) 1.0 (Fourth Edition)”, http://www.w3.org/TR/2006/REC-xml-20060816/.
８．J. Liberty, D. Hurwitz. 2003. Programming Asp.Net, Oreilly & Associates Inc.
９．Microsoft 2010. Microsoft Excel 2003/2007/2010 Help. http://office.microsoft.com/ja-jp/excel-help/ (14 Aug 2010).
１０．W3C. 2001. “XML Schema”, http://www.w3c.org/XML/Schema.
１１．Microsoft. 2002. “Technical Overview of Internet Information Services (IIS) 6.0”, http://
www.microsoft.com/windowsserver2003/techinfo/overview/iis.mspx.

―３０―