Evaluation of an Online Orientation Course for Online Graduate
Students
Junko Nemoto and Katsuaki Suzuki
nemoto@kumamoto-u.ac.jp
Kumamoto University
Kumamoto, JapanABSTRACT
In this paper, we present the result of our trial to develop and deliver an online orientation
course to bring the target incoming graduate students on board to the new learning environment.
We designed an orientation course to focus on the three main factors that relate to their futurelearning: IT, attitude, and planning. We conducted the orientation course for several years insuccession and thus improved our design. To see how students view the learning experience, inthis study, we created a questionnaire that includes the three factors that we embedded in theorientation course and the individual qualities advocated by Parrish, Wilson, and Dunlap. The
results were relatively positive, and we found guidance to improve the orientation course.
Keywords: Orientation Course, Learning Design, Individual Qualities

INTRODUCTION
It is not easy to feel comfortable and be productive in a new environment, and usually people needtime to get accustomed to it. When people move to a new town or change jobs, they need to understandthe role that they play and the procedures in that environment. Like ordinary life, when adjusting to a newlearning environment and preparing to start learning activities, warming up is necessary to meet thecourse expectation. In this paper we report our practice: the provisions of the orientation course forstudents coming into an online graduate program. We conducted the orientation course for several years
in succession and improved our design. To confirm the result of the trial, we created a questionnaire thatincludes the three factors we embedded in the orientation course and the individual qualities advocated byParrish, Wilson, and Dunlap (2010) to see how students view the new learning experience and how it canbe changed into a transformative learning experience.
The graduate program we are teaching (2009, Suzuki) offers an orientation course to prospective
students for the new academic year. Master’s students are expected to take over five courses in the firstsemester; most of them have full-time jobs while being full-time students, therefore, they have not had
student status for some time; some might know how to use the online technology through their work, butsome may not. A successful transition to the new learning environment for the pursuit of a graduate
degree is essential for the learners, and we consider the orientation course as an important first step tobring prospective students into our program.

FIELD INFORMATION
Mission of the Orientation Course to the Target Audience

According to preceding studies, the necessity for an orientation to provide fundamental information
related to the students’ studies is obvious. In addition to the fundamental information, when we designed
the orientation course we included activities to ease the potential anxiety of learning via online and thepotential anxiety from technical problems that can be detrimental to the learning process. This does notmean we are overprotecting the learners. Rather, we expect our students to have confidence and to beautonomous. By having students gather information, think, and then act by them before they call forassistance, we tried to prepare students with a learning experience for the graduate program. Therefore,
we put learning activities as a center part of the course to have students think about and design their own
learning strategies. Our design policy for the orientation course includes three factors in each activity: IT,attitude, and planning (Meta). IT includes practical activities to acquire the operational skills for study;
attitude takes in activities, in which students set a goal to learn according to the new learning environment,curriculum, and with fellow students; Planning (Meta) relates to the activities in which the students set
their own goals and plan how to achieve these goals.
Table 1 shows the structure and activities of the orientation course we designed. The three factors ofactivities included in the course are indicated in the third column. The activities consist of 15 sections,which is the same in all of the courses. The students, therefore, can see how the courses process develops
by taking this orientation course as an example course. They are also able to use all of the fundamental
functions of the LMS (BlackBoard) in our program. Students meet and get to know their peers on line,
and gradually get used to posting their opinions online, willingly and openly. Another important functionof the orientation course is to have students consider their learning goal along with the review of theprogram competencies (Suzuki, 2009).

Individual Qualities
Parrish et al. (2010) advocated a framework that guides the learning experience for an engaging
learning outcome. Their framework proposed two qualities to guide the engaging experience: situationaland individual qualities. Situational qualities have five points: immediacy, malleability, compellingess,resonance, and coherence as critical factors allowing a powerful learning experience to develop. This
helps to design strategies for engaging learning. Individual qualities relate to personal characteristics andinclude four factors: intent, presence, openness, and trust. Individual qualities also influence the learning
outcome, especially online learning and self-directed learning. Parrish et al. implied that the category ofindividual qualities are generally less considered than situational qualities in design process, butinstructions require both perspectives for increasing the quality of learning experiences.
In this study, we focused on the perspective of individual qualities to observe how the students
engaged in learning within the program. Intent includes all of concepts such as attitudes, values, hopes,beliefs, likes and dislikes, and assumptions. Presence has three components: being-there, which includes
physical and mental presence; being-with, which includes willingness to engage with others in a way that
includes empathy and openness to their thoughts and feelings; and being-one’s-self, which includes being
authentic and genuine in expressing one’s own thoughts and feelings. Openness is being willing to submit
to challenge, and it admits to depending on others. Trust includes several qualities of effective experience,
which includes the belief that positive outcomes can occur and that looking ahead to potential outcomeswith engagement will come. The qualities of the individual in the right-most column of Table 1 show theanalytical result of which factor of individual qualities is included, among the four factors, in eachactivity. Most of the sections include one or more factors of individual qualities.

EVALUATION AND RESULTS
Development of the Questionnaires

We developed pre- and postquestionnaires for the orientation course. The design of the questionnaires
are based on the three activities embedded in the orientation course (IT, attitude, and planning) and theindividual qualities of Parrish et al. (2010). After developing the questionnaires, we conducted anexpert review and revised them several times until the reviewer told us to proceed to the next step. Then
we conducted a one-on-one evaluation with students who previously took the course. After smallmodifications, such as wording, we proceeded to the practices.
The questionnaires consist of 19 questions that include all four factors of the individual qualiti
es and the three activities we embedded in the orientation course, as shown in Table 2.

Questionnaire Results

We asked the student participants to fill in the questionnaires in the orientation course. Sixteen
among the 18 participants answered both the pre- and postquestionnaires. The questionnaires have a 5point scale, with 5 as the most affirmative response and 1 as the least affirmative response. Table 2 showsthat the overall score was relatively high. The average score was 4.1 (SD = 0.68) in the prequestionnaire,and 4.3 (SD = 0.56) in the postquestionnaire. The results show that most of the participants were highly
motivated before entering the program, but the results also implied that the orientation course helped
students to get on board by increasing their confidence and volition. The questions on presence (No. 5),openness (No. 7), IT (No. 12), attitude (No. 16), and planning (Nos. 17 and 18) showed significant
differences between the pre- and postquestionnaires.
The scores for No. 6 and No. 9 were higher in the prequestionnaire than the postquestionnaire. Q6
was relatively high in the prequestionnaire. Q9 was reversed question, and, therefore, we feel that wehave not provided clear messages in the orientation course that relate to this question. This is an example
in which the study reflects the lack of clarity within the course, and, thus, assists us to know where weneed to make improvements.
For further research, we need to combine the results, indicated in this paper, with the writtencomments in the questionnaire and the students’ activity record such as their discussion in the orientation
course. An integrative analysis will provide additional concrete ideas to improve the orientation course
for the incoming students. Also, as design researchers, we are continuing our study to deliver designprinciples.

DISCUSSION
Aiming to confirm our trial to develop and deliver an online orientation course that will target theincoming graduate students to bring them on board in their learning experience in a new learning
environment, we developed and implemented pre- and postquestionnaires in view of three main design
factors and the qualities of the individual (Parrish et al., 2010). In the findings of this study, mostparticipants were highly motivated before entering the program, but the result also implies that theorientation course helped students to get on board by increasing their confidence and volition. Conversely,
we found areas within the orientation that need to be improved such as conveying trust in the learning
environment in our graduate school.
Table 1. Structure and Activities of the Orientation Course
Factors of
Qualities ofLearning Activities
Information Provided and Its Resources
Activities
Individual
Mental attitude toward learning in the1Self-introduction
IT/Attitude
I・P・O
program/Instructors /Office hours
2
Task-check operationIT
3
Confirming learning system
ITSystems of the university
I
Information about student ID・setting about
4
Posting a message to mailing list IT/AttitudeI
email address/Mailing list
Planning/ Competencies/Certification of eLearning
5
Create a goal statement
I/P/O
Attitude professionals/Final exam and portfolio
Conditions for completing the degree
Planning/
6Course planning(credits/final exam)/Syllabus/assigned
I
Attitudereading book
7Writing a reflection paper
Planning Same resources as section 5 and 6I/P/O
8
Pedagogical Fundamentals
Planning Condition of course exemption
I
9
Level check of IT skills
Planning Assignment for course exemption
I
Information Processing
Information about “Information Processing
10PlanningI/P
Fundamentals
Fundamentals” as prerequisite course
11 Reviewing past educational activity Planning About learning portfolio
I/P/O
Story-centered Curriculum (SCC)/
12 Expressing an impression of SCC Attitude
I
Advice from the seniors
13 Expressing use of a HOME page IT/Attitude Web pages for SCC
I
14 Requesting of SCC course taking Planning
O/T
Posting to notify the completion of15
Attitude
I/P/O/Tthe orientation course
Twitter of a senior/Schedule/Instruction ofOther
Q&A
All
WebCT (LMS)/Course structure/Tips fromstaff
Note. I = Intent; P = Presence; O = Openness; T = Trust.
SectionNo.
1Intent23Presence

456Openness

789Table 2. List of Questions and the Results
QuestionsTerm
I have a clear idea what knowledge and skills to be obtained at thePregrad school.
Post
I have a clear idea what I would like to obtain with the exception ofPreknowledge and skills at the grad school.
Post
Pre
I have a clear idea what kind of attitude to take toward learning as alearner at the grad school.
Post
PreI would like to contribute to learning activities as one of themembers, but not as a passive learner in the activities in the gradschool.
Post
Pre
I would like to express my thoughts and feelings at the grad school,
even about my deficiencies.
Post
I would like to know various thoughts and ideas during the activities
Preat the grad school.
Post
I would like to give attention to own ideas during the activities at the Pre
grad school.
Post
I do not want to resist changing my thoughts about learning, and,Pre
therefore, accept the changes as opportunities to learn at the gradschool.
Post
Pre
If I have any doubts about learning process and activities at thegrad school, I will step forward after resolving them.
Post

Pre
Post
I have conviction that I can get something important through thePre
11learning activities at the grad school.
Post
I understand what operational skills I need to learn at the gradPre
12school.
Post
ITPre
13 I think I need to acquire operational skills to carry out.
Post
Pre
I will understand and accept the learning environment at the grad
14school.
Post
Attitude 15 I will try to understand and accept the curriculum of the grad school. Pre
Post
Pre
16 I will try to accept my fellow students at the grad school.
Post
Pre
17 I have a clear goal to learn at the grad school.
Post
Pre
Planning 18 I have a course plan (at least for the first year) at the glad school.
Post
Pre
19 I think I can make changes, as necessary.
Post
Note.*p < 0.5, ** p < 0.1. Italic: The original item was in the reverse; the results have been converted.

Trust

10 I trust in the grad school as a place to obtain what I want.

MSD

4.3
4.3
3.9
4.2
4.3
4.4
4.5

0.4
0.5
0.8
0.50.7
0.5
0.5

4.7
4.4
4.8
4.9
4.8
3.7
4.3
4.8

0.5
0.5
0.4
0.3
0.4
0.9
0.7
0.4

4.8
2.4
2.3
4.4
4.5
4.6
4.6
3.3
3.8
4.3
4.2
4.5
4.4
4.4
4.4
4.4
4.8
3.9
4.3
2.6
4.1
3.6
3.9

0.4
0.8
1.1
0.7
0.5
0.5
0.6
0.9
1.0
0.9
0.9
0.5
0.8
0.5
0.5
0.5
0.4
0.7
0.8
1.2
0.6
0.9
0.7

REFERENCES
Parrish, P., Wilson, B., & Dunlap, J. (2010). Learning Experience as Transaction: A Framework forInstructional Design. [Retrieved fromhttp://homes.comet.ucar.edu/~pparrish/papers/LearnExp%20As%20Trans%20_final.doc]
Suzuki, K. (2009). From Competency List to Curriculum Implementation: A Case Study of Japan's First
Online Master’s Program for E-Learning Specialists Training. International Journal on E-Learning,
8(4), 469-478. Chesapeake, VA: AACE.