Journal of Interactive Online Learning
www.ncolr.org/jiol

Volume 6, Number 3, Winter 2007
ISSN: 1541-4914

Online vs. Traditional Course Evaluation Formats: Student Perceptions
Judy Donovan
Indiana University Northwest
Cynthia Mader and John Shinsky
Grand Valley State University
Abstract
The decision on whether to offer end-of-course evaluations in an electronic format or in
traditional scan sheet format generates conflicting viewpoints. From an expediency perspective,
offering evaluations online saves time and money and returns the results to faculty more quickly.
From a student point of view, concerns involve convenience and anonymity. This study examines
the issue from the student viewpoint to identify opposition, support, concerns, and preferences
for each format. An analysis of the results reports commonalities and differences in responses
based on variables such as gender, experience with online evaluations, and program level.
Analysis also draws conclusions about improving the use of end-of-course evaluations.
Introduction
Among instructional areas in higher education, few areas cause more interest than course
evaluations, and few areas have been studied more for validity and reliability (Wachtel, 1998).
From the viewpoint of students, end-of-course evaluations provide the opportunity to give thefeedback that faculty find so helpful. Although some students may harbor doubts about theeffectiveness of evaluations, few would abdicate the opportunity to pass judgment on the courseand suggest improvements. Course evaluations provide a way for students to lodge a complaint,
extend a compliment, express appreciation, improve a course, and most importantly, participate
in the learning community.
At a time when incorporating online methods, even into on-site courses, is no longer anovelty, online course evaluations bring the advantage of saving time and resources overtraditional paper and pencil scan sheet method. The research questions posed in this study werethese:
1. Which evaluation format is preferred by students and why?
2. Are format preferences and reasons related to program level, gender, and prior online
evaluation experience?
3. What can we learn about improving students’ experiences with faculty evaluationsdelivered online?
Perspectives from the Literature
Students, administrators, support staff, and faculty all have a stake in whether student
evaluations of faculty are delivered online or in the traditional paper and pencil manner.
Administrators may wish to save money, in the form of paper Scantron sheets, machines to score
evaluations, pencils, and most importantly, staff time. Support staff may spend days, even weeks,scanning, tabulating, and collating evaluations (Dommeyer, C., Baum, P., Hanna, R., &158

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

Chapman, K., 2004). Staff may need to aggregate quantitative scores, retype comments so thatstudent anonymity is retained, distribute and collect the evaluations, and prepare bothadministrative and faculty hard copies (Dommeyer et al., 2004).
It may be inconvenient for adjunct faculty and weekend faculty to return the evaluations
to the proper office in a manner that maintains confidentiality, as the main office may be closed.
In many classes, evaluations are often delivered the last night of class. The procedure has faculty
passing out the evaluations, reading the instructions to the students, and then leaving the room.
Students complete the evaluation and one student is tasked with taking the completed evaluations
to the proper office. For night and weekend faculty, the office is closed and inaccessible. At the
research site, the office is located on an upper floor, and for security purposes, the elevator does
not go to that floor after hours. Faculty or a student may have to hold onto the evaluations for a
day or more and take them to the office when it is open, jeopardizing the confidentiality of theprocess.
The benefits of having students complete faculty evaluations online as compared to the
traditional paper format include time and cost savings and faster reporting of results (Kuhtman,2004). Often it takes months for staff to return traditional evaluation results to faculty, by whichtime it may be too late to implement changes based on students’ comments and scores. Online
results can be made available to faculty within days instead of weeks. For these and otherreasons, many institutions are moving toward or have implemented online or electronic student
evaluations of faculty.
While students may not be as interested in saving time and money as administrators or
support staff, they have reasons of their own for preferring one form of evaluation delivery toanother. Students may perceive either the traditional or the online format as more anonymous,and research indicates anonymity to be important to students (Carini et al., 2003; Dommeyer etal., 2004). Some research has shown online evaluations are less susceptible to faculty influence(Anderson, Cain, & Bird, 2005; Dommeyer, Baum, Chapman, & Hanna, 2002). Some students
perceive online evaluations as more anonymous (Ravelli, 2002).
Students may appreciate the flexibility of online evaluations; they are able to take theevaluation at a time of their choice and spend as much or as little time as they choose completingthe evaluation. Online surveys may result in more open ended responses or comments because
students have additional time available (Dommeyer et al., 2002). Students believe the online
format allows them to provide more effective and more constructive feedback than the traditional
paper format (Anderson et al., 2005; Ravelli, 2002). Recent research supports this belief as
online evaluations generate substantially more constructive student comments (Donovan, Mader,& Shinsky, 2006).
Research reports that students prefer completing online faculty evaluations to paper ones
(Dommeyer et al., 2004; Layne, DeCristoforo, & McGinty, 1999). In the Anderson survey, over
90% of students marked Agree or Strongly Agree when asked if they preferred online to
traditional evaluation format (Anderson et al., 2005).
Online evaluations have disadvantages, such as requiring students to have computer
access, students needing to know their login information, and students experiencing technicalproblems when accessing the evaluation (Anderson et al., 2005). Online evaluations usually havelower student response rates as students may not take the time or remember to complete theevaluation if it is not done in class (Laubsch, 2006). Sax, Gilmartin, and Bryant (2003) identified
several concerns with online evaluations, such as the fact that lengthy online survey instruments

159

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

have fewer responses than shorter ones, and students may reach the saturation point and not wantto fill out additional online surveys.
Research says little about the preferences of different types of students. One study looked
only at undergraduate students and concluded that more affluent, younger males are more likely
than comparable females to complete online surveys as compared to paper ones (Sax et al.,2003). Thorpe (2002) looked at gender, minority status, grade received in the class, and grade
point average in three computer science, math, and statistics classes. Results showed gender was
significant, and that women were more likely than men to complete the online evaluation ascompared to the traditional in-class paper evaluation (Thorpe, 2002).
A comprehensive survey to identify and clarify student preferences was designed toaddress some of these issues. The goals were to discover not only which evaluation delivery
method students prefer, but the reason for these preferences. The researchers also sought todiscover differences in responses between students by gender and program level.
Research Methods
A study involving College of Education students were conducted at a large Midwesterncomprehensive public university. Participants were drawn from the same college within theuniversity in order to ensure common course experiences and evaluation forms.
The college uses Blackboard course sites to administer online student evaluations offaculty. One site is created by the Blackboard administrator for each class for which the faculty
indicated they wished to deliver the evaluation online. This ensures confidentiality, as only
students can complete or access the evaluations. The identical evaluation instrument is used forboth traditional and online formats. Faculty are free to choose the format (traditional or online)
they prefer, and how they administer the evaluation (in class or allowing students to complete theonline evaluation on their own). Traditional evaluations are usually given to students during thelast class session. The online evaluations are available for a 2-week period near the end of thesemester. Some faculty take students to a computer lab during the last class session to complete
the online evaluations, but most simply tell students the evaluations are available online and ask
them to complete them before the class ends.
All students who had taken at least one class in the college during the previous year weree-mailed a link to an anonymous electronic survey during the summer of 2006. This survey wascreated and administered using the online survey software SurveyMonkey (SurveyMonkey.com).
Students who did not respond to the original e-mail within a week were sent a second request.
Student surveys were returned by 851 individuals, from a total of 4052 sent. All 4052 email
addresses were generated by the university database, although many (approximately 250) emails
were returned as not deliverable. Thirty-eight students declined to respond.
The survey asked students to indicate their faculty evaluation format preferences, reasonsfor the preferences, and demographic data. The survey also contained a section for open-ended
responses. The results were then analyzed to determine students’ preference for online or
traditional evaluations. The data were further analyzed to see if undergraduates varied from
graduates in their preference, males or females preferred one evaluation format to another, and if
students who have completed more evaluations online prefer the online or traditional format. The
researcher asked students the reasons for their preferences, and their responses illuminated thesurvey results and described implications for practice.

160

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

SurveyMonkey is a powerful online survey tool which allows users to analyze results ofsurveys in several ways. One of these methods involves filtering (cross tabulating). Using thefilter feature, researchers were able to determine the response of males or females and graduate
or undergraduate students, as well as compare variables such as experience with online
evaluation to location preference. The survey allowed for general comments and student
comments directed at specific survey questions, such as student preference regarding where theonline survey is delivered (in a computer lab or on their own outside of class). Comments were
filtered and grouped by student gender, program level, experience with online evaluations, or any
combination of the variables. Therefore it was possible to isolate, for example, the comments offemale undergraduates who had completed online evaluations three to four times and who
preferred to complete the evaluations in the traditional format. The data analysis tools in
SurveyMonkey allowed researchers to analyze and report both quantitative and qualitative datafrom over 800 respondents accurately and quickly.
Results of Student Surveys
Table 1 provides a picture of the students who completed the survey. Two demographic
questions were asked at the beginning of the survey regarding program level and gender.
Table 1: Demographics (N=831)
Between Program Levels

Between Genders

Undergrad
N=431

Graduate
N=397

Male
N=161

FemaleN=669

Undergrad
Male N=90

UndergradFemaleN= 341

Grad Male
N=70

Grad Female

48.1%

19.4%

80.6%

10.8%

41.0%

8.4%

39.2%

51.9%

Within Program Level, Within Gender

N=326(Note: Percentages will not always equal 100% because not every respondent responded toevery question.)
Demographic data show that graduate and undergraduate program levels were almost
equally represented among the respondents, with slightly more undergraduate respondents than
graduate. Females greatly outnumbered males, the usual case for Colleges of Education, which
serve mostly pre-service and in-service teachers.
To address the second research question about differences in student responses based on
gender and program level, the researchers wanted to find out if graduate or undergraduate
students, and male or female students had more opportunity to take online evaluations. In
addition, researchers wanted to know how many took advantage of opportunities and actuallycompleted online faculty evaluations. Table 2 and 3 show the student responses addressing thisissue.

161

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

Table 2: Opportunities to complete course evaluations online (N=828)
Between
Program Levels
Number ofOpportunities

Overall

Between
Genders

Within Program Level,
Within Gender

FemaleUndergrad
Male

UndergradFemale

Grad
Male

Grad
Female

5.6%

3.7%

5.6%

1.5%

5.7%

6.1%

42.3%

26.2%

32.7%

13.3%

23.5%

42.9%

42.3%

31.6%

21.9%

25.0%

27.3%

27.8%

32.6%

21.4%

21.8%

44.8%

29.7%

43.1%

36.3%

53.3%

42.5%

30.0%

29.8%

Response

Undergrad

Grad

Male

0 times

4.1%

2.3%

6.0%

1-2 times

31.4%

21.3%

3-4 times

26.9%

5 + times

37.6%

Table 3: Actual Completion of Course Evaluations Online (N=830)
Between
Program Levels

Number ofCompletions

Overall

Between
Genders

Within Program Level,
Within Gender

FemaleUndergrad
Male

UndergradFemale

Grad
Male

Grad
Female

9.4%

8.0%

4.4%

3.2%

15.7%

12.9%

41.2%

26.2%

33.2%

16.7%

25.2%

38.6%

41.5%

32.3%

22.2%

30.0%

26.9%

34.4%

31.7%

24.3%

21.8%

40.8%

23.2%

34.4%

32.0%

44.4%

39.9%

21.4%

23.7%

Response

Undergrad

Grad

Male

0 times

8.2%

3.5%

13.4%

1-2 times

31.9%

23.4%

3-4 times

27.3%

5 + times

32.5%

Overall. Almost all respondents had at least one prior opportunity to complete an end-ofcourse evaluation in online format. Only 4.1% overall reported that they never had theopportunity to complete a course evaluation online.
When asked about actual completion of online evaluations (rather than opportunities tocomplete), only 8.2% of respondents reported never having completed an online evaluation.
These respondents would comprise the 4.1% who never had an opportunity, along with another
4.1% that had an opportunity but had not followed through. The remainder of the respondents
were almost evenly divided between 1-2 completions (31.9%), 3-4 completions (27.3%), and 5
or more completions (32.5%).
Program Level. Comparative results by program level, however, show that
undergraduates had more opportunities than graduate students to complete evaluations online.
The largest undergraduate response showed that 44.8% had at least 5 opportunities, compared toonly 37.6% of graduate students. Conversely, more than twice as many graduate students (6%)
reported never having an online opportunity, compared to the percentage of undergraduates who
never had an online opportunity (2.3%).
162

Journal of Interactive Online Learning

Donovan, Mader, ShinskyComparative results by program level show that far more undergraduates had actuallycompleted online evaluations than graduate students. Over 40% of undergraduates hadcompleted at least five online evaluations, compared to only 23.2% of graduate students who hadcompleted that many. Furthermore, only 3.5% of undergraduates had never completed an online
evaluation, compared to 13.4% of graduate students. In fact, program level data show virtual
reverse images of each other with over 40% of graduate students completing only one or two
online evaluations, compared to a little over 23% of undergraduates. Conversely, over 40% of
undergraduates had completed at least five online evaluations, compared to a little over 23% ofgraduate students.
Gender. Differences in opportunity were also apparent by gender at both the high andlow ranges. In the high range, over 43% of all male students had at least five opportunities tocomplete course evaluations online, while only slightly more than 36% of all female students
experienced that many opportunities. In the low range, however, more males (5.6%) thanfemales (3.7%) also reported having no opportunities to complete online evaluations.
Differences in actual completion of online evaluations were also apparent by gender, withmale students completing in greater percentages than female students. Although genderdifferences were less apparent when students were asked about completion than when asked
about opportunity, 34.4% of males reported completing at least five evaluations, compared to32.2% of females
Program Level and Gender. Undergraduate males and females reported more
opportunities than graduate males and females did, especially in the high ranges of five or more
opportunities. Gender differences were least within the graduate level, with less than a 1%
difference in each opportunity range. When looked at together, the greatest variability came
between genders at both levels, with the range for males at both levels at 29.5% (0.1%-29.6%)
and for females at 14.2% (4.6%-18.8%). Less variability was seen between programs by bothgenders, with the range for undergraduates at 6.1% (4.1%-10.2%) and the range for graduate
students at 0.4% (0.2%-0.6%).
When program and gender were examined together, undergraduate males showed not
only a higher completion rate than undergraduate females but also higher than graduate studentsof either gender. Similarly, more male undergraduates completed five or more evaluations(44.5%) than females did (39.9%). Gender differences were least apparent at the graduate level,with less than a 3% male/female differential at each completion range.
The next survey question asked students to indicate whether they prefer to take the online
evaluation on their own by signing in to Blackboard and competing the survey, or if they prefer
the class to go to a computer lab together and complete the evaluation. At the university understudy, instructors who offer online evaluations offer one of two locations for students tocomplete them. Some instructors take the class as a group to a computer lab to complete theevaluations, usually during the last class meeting, and others ask that students complete them attheir convenience in a location of their choice during the 2-week period the evaluations areavailable online. Table 4 summarizes students preferences overall in terms of location,
preferences by program level, gender and by program level and gender combined.

163

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

Table 4: Location Preference for Online Evaluations (N=826)
Between
Program Levels

Location
Preference

Between
Genders

FemaleUndergrad
Male

UndergradFemale

Grad
Male

Grad
Female

84.4%

84.6%

88.9%

85.3%

78.6%

83.7%

15.6%

15.4%

11.1%

14.7%

21.4%

16.3%

Response

Undergrad

Grad

Male

On Own
Time

84.6%

86.1%

82.9%

Computer
Lab

15.4%

13.9%

17.1%

OverallWithin Program Level,
Within Gender

Overall. A majority of students preferred to complete online evaluations on their own
time rather than as a group in a university computer lab. Some 84.6% chose “on own time,”
compared to 15.4% who preferred the group lab setting.
Program Level. At the program level, a greater percentage of undergraduate students
favored completing evaluations on their own time rather than as a class in a computer lab,
although the difference between graduate and undergraduate students was only 3.2%. Some
86.1% of undergraduates preferred the choice option, compared to 82.9% of graduate students.
Gender. The gender difference in location preference was a mere 0.2%, with females
expressing a slightly stronger preference than males for completing the evaluation on their own
time as compared to in a computer lab with their class.
Program Level and Gender. When gender and program level were looked at together,location preferences showed a similar pattern to earlier responses, although, unlike earlier
responses, the effect of program was less than the effect of gender. Although all students
preferred completing evaluations at their own convenience, and although over 3% more
undergraduate males than females reported this preference, over 5% more graduate females
preferred this option than graduate males. Male graduate students, more than any other group,
prefer to complete the evaluation as a class in a computer lab.
Comments Favoring Completion on Own Time.
(a) I am more likely to write specific comments while I am relaxed at homeinstead of in a lab and anxious to leave.
(b) The issue of confidentiality is also raised when the whole class goes to thecomputer lab. It is easy to see what is written on nearby computer screens. I do
not want to be concerned that fellow students may be reading portions of what I
write in a confidential evaluation. Even fleeting access or the possibility of accessis unfair to me and to the professor.
(c) I would feel uncomfortable and constrained to keep my eyes glued to myscreen to avoid the appearance of glancing over at what someone else is writing.
(d) I had one class that spent time to go to the computer lab to fill out anevaluation. It was a significant waste of my time. I should not have to pay almost$900 per class and then be required to spend two hours of it evaluating the
instructor.
164

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

Comments Favoring Completion during Class Time in Computer Lab(a) It would be very helpful for the professor to allow time in the computer lab for all
students to complete the survey. This way I won’t forget and there will be otherclassmates present to help if need be.
(b) I have found that if the whole class does not go over together and complete surveys, I
go home and forget to do it.
Overall, of the 128 students who indicated they preferred taking the online evaluation in acomputer lab together as a class, 13 students made comments in the area set aside for thisquestion, and 7seven of the comments pertained directly to the location question. Four students
said they will forget to complete the evaluation if they do not do it in class, one said she needed
help completing it, one said she would not find time to complete it outside of class, and one said
he was not motivated enough to compete the evaluation outside of class.
The survey results indicate about 85% of students polled would rather complete theevaluation at a time of their choosing. Fifteen percent is a significant minority, however, who
prefer to complete the evaluation as a class in the computer lab. Male graduate students inparticular, prefer to complete the evaluation as a class in the lab (21%). Only one male graduate
commented on why he preferred completing the evaluation in the computer lab as a class and hewrote:
It is very difficult for me to motivate/remind myself to complete online course
evaluations. I understand the effort to move towards a paperless process to save money
and time, but it is much easier for me to complete the traditional paper evaluations inclass.
All male graduate students agreed with the statement “I am more likely to complete it since weare already in class.”
Nearly 24% of the students who have never completed the evaluation online prefer tocomplete it as a class in the computer lab, as compared to the 13% who have completed an
online evaluation five or more times and prefer completing the evaluation in the computer lab.
The comments from students indicate the whole class in lab option may help those who havenever taken an online evaluation, female students who have had less experience with online
evaluations, and male graduate students who feel they tend to forget to take the evaluation if theclass does not complete it together.
Table 5: Overall Format Preference (N=826)
Between
Program Levels

FormatPreference

Between
Genders

FemaleUndergrad
Male

UndergradFemale

Grad
Male

Grad
Female

88.7%

88.2%

93.3%

92.4%

82.9%

83.9%

11.3%

11.8%

6.7%

7.6%

17.1%

16.1%

Response

Undergrad

Grad

Male

Online

88.4%

92.5%

83.8%

Traditional

11.6%

7.55

16.2%

OverallWithin Program Level,
Within Gender165

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

Overall. A majority of respondents expressed a clear preference for the online course
evaluation format over the traditional format. When asked which delivery method was preferred,
88.4% of all respondents preferred the online evaluation format, compared to 11.6% who
preferred the traditional format.
Program Level. Although students at both levels expressed a clear preference for online
evaluations, 92.5% of undergraduates favored them, compared to 83.8% of graduate students.
Gender. Differences in format preferences by gender were far less, however, thandifferences by program. Although a greater percentage of males chose online evaluations, malepreferences varied from female preferences by a mere 0.5% for each choice.
Program Level and Gender. Greater differences were apparent when gender and program
were looked at together. Although gender differences alone varied by only 0.5%, the variation
was closer to 10% when gender was combined with program level. Undergraduate males
preferred online evaluations by 10.4% over graduate males. Undergraduate females preferredthem by 8.5% over graduate females. Within programs, the two genders varied from each otherby no more than 1%.
Comments Favoring Online Format. General comments made by students illuminate
their preference for online formats.
(a) (With traditional evaluations), everyone just wants to get out of class so they
bubble in anything and leave without making comments.
(b) (With traditional evaluations) People just fill in all above average ratings justto get it done.
(c) My feeling is that a lot of students simply fill in the paper evaluations and givea false picture of how the course was run. With an online evaluation you may
have fewer students doing them, but they are students that are going to answer
truthfully.
(d) In a computer world, it just makes more sense to do everything online.
(e) We are in a technological age. There is no reason not to utilize this as much as
possible.
(f) I prefer to use the computer.
(g) I hate filling in those stupid bubbles with my name, student id number, etc.
That is a waste of time.
Comments Favoring Traditional Format. Of those who preferred traditional paper
evaluation, about 30% were students who had had no previous experience with online formats.
Of all students who favored traditional formats, 100% agreed with this statement: “I am more
likely to complete it since we’re right in class.”
The reasons students indicated for their choice of evaluation delivery format are further
explored in Tables 6 and 7. Students were given a list of statements (such as “I like completing it
at my own convenience”); see Appendix for complete list and asked to indicate which theyagreed with. Students were also given space on the survey for comments addressing thisquestion.

166

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

Table 6: Reasons If Preference is for Online Format (N=724)
Between
Program Levels

Between
Genders

Reasons forOnlinePreference

OverallResponse

Undergrad

Grad

Male

FemaleWithin Program Level,
Within Gender
UnderUndergrad
grad
Grad
Grad
Male
FemaleMale
FemaleConvenience

87.2%

88.2%

85.8%

85.0%

87.6%

87.7%

88.4%

81.0%

86.7%

Anonymity(typed)

58.1%

61.1%

54.5%

49.6%

60.2%

53.1%

63.2%

44.8%

56.8%

Privacy

54.4%

55.8%

52.4%

40.3%

57.5%

43.2%

59.0%

36.2%

55.7%

Time to
Think

53.6%

58.3%

47.6%

48.2%

54.6%

55.6%

59.0%

37.9%

49.4%

Write More

49.6%

55.0%

43.0%

84.9%

87.6%

45.7%

57.4%

43.1%

43.2%

Doesn’t
Take
Class Time

48.2%

46.0%

50.9%

49.6%

48.2%

44.4%

46.5%

56.9%

49.4%

Other

9.5%

9.5%

9.7%

8.6%

9.8%

8.6%

9.7%

8.6%

10.0%

Table 7: Reasons If Preference is for Traditional Format (N=93)
Between
Program Levels
OverallResponse

Undergrad

Grad

Male

FemaleWithin Program Level,
Within Gender
UnderUndergrad
grad
Grad
Grad
Male
FemaleMale
FemaleMore Likely
to Complete

94.6%

90.3%

96.8%

93.8%

94.8%

75%

92.6%

100%

96.0%

More
Familiarwith Format

36.6%

41.9%

33.9%

18.8%

40.3%

50%

40.7%

8.3%

40%

Hard to Get
OnlineAccess

30.1%

29.0%

30.6%

6.2%

35.1%

0%

33.3%

8.3%

36%

Can Ask for
Help if
Unclear

18.3%

22.6%

16.1%

12.5%

19.5%

25%

22.2%

8.3%

18%

Other

21.5%

22.6%

21.0%

25.0%

20.8%

50%

18.5%

16.7%

22%

Reasons for
Traditional
Preference

Between
Genders167

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

Overall. Convenience was the first choice of those who preferred online evaluations; it
was chosen by at least 87.2% of respondents. The next most common reasons, in descendingorder, were anonymity, privacy, and more time to think before writing the comments. These
were all chosen by over half of respondents. Almost half of all respondents also said that they
were able to make more comments when typing than when writing by hand, and almost half
preferred online evaluations because they didn’t take class time. “Other” reasons were added by
9.5% of respondents.
Although only 93 students preferred the traditional format, they were united in theirreason. Almost 95% said that they were “more likely to complete” traditional evaluations, which
are customarily delivered in the classroom on the last night of class. Other reasons included
familiarity with traditional format, difficulty of getting online access, and ability to ask for help
if questions are unclear. Interestingly, some 21.5% chose “other” as their reason, and some of thecomments listed below will illuminate this choice.
Program Levels. Some differences in rankings emerged according to respondents’program levels, although the differences were not large. Both graduate students andundergraduates appreciated the convenience and anonymity of online evaluations, although
graduate students, more than undergraduates, appreciated not taking class time to complete
evaluations.
Among those who preferred the traditional evaluation format, an analysis of the responseby program level showed similar preferences from both groups, although over 6% more
graduates than undergraduates chose “more likely to complete” as their reason for preferring
online evaluations. Over 6% fewer graduate students placed value on being able to ask for help
than undergraduates did.
Gender. When analyzed by gender, males and females both chose convenience as themost common reason for liking online evaluations, with females showing a slightly stronger
preference. Both males and females said they were able to make more comments when typing
than when handwriting. The biggest difference was more females reporting a much stronger
preference because of anonymity and privacy. Compared to males over 10% more females
identified anonymity, and over 17% identified privacy as their reasons for preferring online
evaluations.
Among those who preferred traditional formats, the spread between male and female wasmuch greater than that between program levels. For example, 21.5% more females than males
cited format familiarity as a reason for preferring traditional evaluations, and 30% more females
than males said that getting online access was problematic.
Program Level and Gender. Within program levels and within genders, convenience wasagain cited most often, and especially by females at both levels. Between 10% and 19% more
females at both levels also appreciated privacy and anonymity, whereas these aspects seemed
less important to males. Privacy, in fact, was ranked last in importance by the fewest males atboth levels.
Increased likelihood of completion was cited within both genders and within bothprograms as the most common reason these 93 students preferred traditional evaluations. Indeed,
all 100% of male graduate students cited this as their reason, with other reasons cited by only
8.3%. Far fewer male undergraduates seemed to value likelihood of completion, however, with
only 75% choosing it compared to over 92% in each of the other groups. A full 50% of maleundergraduates chose “other.”168

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

Comments on Reasons for Online Preferences(a) Don’t have to worry about turning it in.
(b) Easier to click than fill in bubbles.
(c) You don’t feel as if other students are waiting or watching to see if you have a
lot of comments about the teacher or just a few comments. It’s more private if you
really have thoughts about a teacher you want to include.
(d) I feel that students are much more honest when doing evaluations online . . . I
know that even though the teacher wasn’t able to see the evaluations, I still was
worried about it getting into his/her hands somehow. I hope we never go back tothe bubble sheet!
(e) I still feel (online evaluations) aren’t quite anonymous, though, because if I
have something bad to say when the professor sees these he/she can still figureout who it was. So most of the time I don’t type messages, I just do the check box
part. I feel it is hard to be truly honest knowing the professor will see theevaluation.
(f) I am more thorough with my evaluations and am able to provide concrete
examples of what I liked and disliked about the class . . . I found that I have moresuggestions to offer with the evaluation being online.
(g) (Online surveys are) not as big of a problem with teachers forgetting about theevals until the last three minutes of class, and then forcing you to rush throughthem.
Comments on Reasons for Traditional Format Preference(a) Online surveys are always posted during the busiest time of the semester(exams), and I usually don’t have time to fill it out unless it’s completed duringclass. Completing surveys during my own time is at the very bottom of mypriority list even when I really do want to give my opinion on my class andprofessor.
(b) I don’t have to take the time outside of my class, which I do not want to do.
(c) Computer input can be traced, and students know this. Online evaluation is nota secure channel, in my opinion. I am skeptical that my input could not be
accessed.
(d) Although supposedly anonymous, online surveys have to be able to be
tracked. A bubbled sheet allows more anonymity and more honesty.
(e) I prefer to complete evals on the bubble sheet because this way I know the
responses cannot be linked to me. (In online surveys) I know that someone
somewhere can connect my responses to my name. This has actually kept mefrom completing the course evaluation for two professors, but I would havecompleted them if they had not been online.
The comments students wrote indicate a variety of concerns with online evaluations,especially in the areas of anonymity. Students comments support previous research indicatingthat students write far more comments when the evaluations are online, and perceive their
responses as more honest (Anderson et al., 2005; Dommeyer et al.,.2002; Donovan et al., 2006)
The next area investigates the relationship between students’ experiences with competing
evaluations online and their preferences for online or traditional formats. Does more experience
competing evaluations online tend to alleviate students concerns and allow them to prefer this169

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

method? Table 8 compares experience with completing online evaluations with preference fordelivery format across program levels and gender.
Table 8: Actual Completion of Course Evaluations Online (N=830)
Between
Program Levels
Number ofCompletions

Overall

Between
Genders

Undergrad

Grad

Male

Female

22.1%

77.9%

22.1%

1-2 times

68 / 8%2
65 / 32%

38.1%

61.9%

3-4 times

227 / 27%

61.2%

5 + times

270 / 33%

65.2%

Response /Preferences Within Program Level,
Within Gender
Online Preferred

Traditional
Preferred

77.9%

69.1%

30.9%

15.9%

84.1%

17%

38.8%

21.1%

78.9%

83%
9
4.2%

5.8%

34.8%

20.7%

79.3%

94.1%

5.9%

Percentage
0 times

Overall. One area which intrigued the researchers was the difference in student responses
based on familiarity with online evaluations. If students had used the online format to evaluate
faculty extensively in the past (defined as five or more times), were they more or less likely toprefer the online format over the traditional pencil and paper? For students with no experience
completing faculty evaluations online, which format did they prefer and why?
Inexperienced students. Sixty-eight student respondents, 53 graduate students and 15
undergraduate students, indicated they have never completed a faculty evaluation online. As
shown in Table 3, 13% of graduate students had not completed an online evaluation, while only
3.5% of undergraduate students never had this experience. Survey results showed 69% of thestudents who had no experience with online evaluations still indicated they would prefer tocomplete faculty evaluations online rather than in the traditional bubble sheet format. The
reasons for these students’ choices were fairly consistent. The students felt rushed completing
evaluations in class. Comments such as these were representative of this group:(a) In the past when I have completed evaluations in class, it is always done in arush, and there seems to be pressure to finish quickly.
(b) I like not feeling rushed to turn in the evaluations while in class. I like to takemore time filling them out than most students.
(c) Less paperwork... I know it's the same exact thing, but it's seriously annoying
to have to hand out those papers the last 10 minutes of class.
About 30% of students with no experience with the online format preferred the traditional
format. One hundred percent of these respondents indicated agreement with the statement “I am
more likely to complete it since we’re right in class.” Perceived barriers to online format are theinability to ask for help while completing the evaluation, the need to have consistent online
access to complete the evaluation, and the lack of familiarity with the online evaluation format.
Student comments were varied and interesting in this area, as indicated below:(a) I don't like the --SU practice of not requiring students to take responsibility for
their evaluations. Like other state universities I have attended, students should be170

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

required to sign their evaluations if the evaluation is going to go beyond theprofessor. Online does not give me the opportunity to sign or take responsibility
for my comments.
(b) I don't have to take the time outside of my class, which I do not want to do.
(c) I like the convenience of the Bubble Sheet in class. The main complaint I haveis that both of my professors that offered the Bubble sheet passed it out in the last
5 minutes of class, giving no warning so I was unprepared. If it was not possible
to stay late during that time to finish it, the eval would be done very poorly, which
renders it useless.
Students with a great deal of experience with online evaluations. Two hundred andseventy students indicated they have completed the faculty evaluation online five or more times.
Sixty five percent or 176 of these were undergraduate students, and 35% graduate students, again
reinforcing the finding that online evaluations use appears more prevalent in undergraduate
classes. This experienced group of students indicates that they would rather complete the facultyevaluations on their own time (87%) as opposed to in a computer lab with the entire class (13%).
This is higher than students who have never completed an online evaluation; for these 76.5%
would choose to complete the evaluations on their own time, but 23.5% would rather complete
them in a lab with the whole class. It appears as if once students are comfortable with the format,
they do not need the support of the instructor or whole class lab setting. Findings indicate themore experience students have completing faculty evaluations online, the more they prefer the
online format. 70% of inexperienced students preferred the online format as compared to 94% of
the experienced students.
Experienced who prefer online. Experienced students preferred the online format
primarily because of convenience – 90% agreed that they like completing the evaluation when it
is convenient to them. Fifty eight percent liked having more time to think before writing andliked the privacy and anonymity of online evaluations. Over half, 54%, indicated they can write
more comments when they complete the evaluation online. Some of the student comments
include:(a) I hate filling in those stupid bubbles with my name, student id number, etc.
That is a waste of time(b) Teacher's can't alter the process or ask a "friend" to administer the evals andthen give specific questions for you to answer in order to cover up the class'dissatisfaction with their teaching. That happened to our class last semester and
we all were upset about how it was handled in the classroom. We left feeling likeour right to voice our opinion was meddled with.
(c) There is less hassle, since a student must take the bubble sheets to thedepartment office after everyone has completed the evaluations in class.
(d) I like that I can take it right away so my thoughts are right after completing theclass. I think you get a more accurate response.
(e) When completing surveys in class, it seems all students are compelled to "doso quickly" as it is often handed to you right as you leave the class. On the other
hand, on-line surveys, although my preference, can be frustrating if Blackboard is
not functioning properly.
(f) I am more likely to write specific comments while I am relaxed at homeinstead of in a classroom and am anxious to leave.
171

Journal of Interactive Online Learning

Donovan, Mader, Shinsky(g) I am more thorough with my evaluations and am able to provide concrete
examples of what I liked and disliked about the class by looking back at the work
I have completed. I found that I have more suggestions to offer with theevaluation being online.
Experienced who prefer traditional. Students with a lot of experience with online
evaluations, who prefer traditionally delivered evaluations, did not make as many comments.
The one student (of the 12 who indicated this choice) wrote:
I know that it takes longer to code the data. I happened to be in a department's office thatused the online format and they were giving the evaluations to the professors before final
grades were due. I do not think that professors should see the evaluations until the grades
are in!
Generally, experienced student responses agreed with the inexperienced students as 75%
indicated they were more likely to complete the evaluation in class than online. As compared to
inexperienced students of whom 43% indicated more familiarity with the traditional format, 32%of the experienced students liked the familiarity with the bubble sheet format. About the same(285 of inexperienced students as compared to 25% of experienced respondents) who preferred
the traditional delivery format were concerned about online access. The biggest difference
between the two groups was that the only 12.5% of the experienced students as compared to 24%of the inexperienced students indicated the fact that they could ask for help mattered to them.
Discussion and Implications
Several conclusions can be drawn from this research, though it should be replicated inother settings with other populations. In answer to the first research question, “Which evaluation
format is preferred by students and why?” students clearly and overwhelmingly prefer tocomplete faculty evaluations online as compared to the traditional pen and paper format. This
holds true regardless of program level, gender, experience, or any other factor looked at in thisresearch. The percentage of students who indicated a preference for the online format was 88.4%
overall, and depending on the subgroup, ranged from 82.9% (male graduate students) to 93.3%(male undergraduates).
The second research question was, “Are format preferences and reasons related toprogram level, gender, and prior online evaluation experience?” Format preferences are related
to program level as a higher percentage of undergraduate students prefer online evaluations than
graduate. Format preferences are not strongly related to gender as approximately the same
percentage of male and female students prefer online evaluations, although a deeper analysis
reveals minor differences in responses between male and females. Format preferences are related
to students’ prior experience with online evaluations, as students with more experience prefer the
online format.
The survey results show more graduate students have never had the chance to complete
an online evaluation (6% of graduate students as compared to 2.3% of undergraduate students),
and 13.4% had never completed the evaluation online, as compared to only 3.5% of
undergraduate education students. Forty percent of undergraduates had completed at least five
online evaluations, compared to 23.2% of graduate students. Undergraduate students have had
more opportunity and experience with online evaluations than graduate students, and this helps
explains why 93% of undergraduate students prefer online evaluations as compared to 84% ofgraduate students.
When student experience with online evaluations is compared with student preference,the preference for online delivery increases as experience with this format increases. Of students
172

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

with no experience taking online evaluations 69.1 % prefer the online format, but as experience
increases to three or more times, 94% of students choose the online delivery format. Conversely,
30.9% of students with no experience taking online evaluations prefer the traditional delivery
format, but as experience increases this decreases to 5.8% of students who have completed
online evaluations three or more times preferring traditional delivery.
The level of experience affects student’s choice of location for completing the evaluation.
When experience with online evaluations was looked at in relation to preference of location,some interesting trends emerged. Eighty seven percent of experienced students, those who havetaken five or more evaluations online, prefer completing them on their own time, which isslightly higher than the overall student response of 85%. This compares to 77% of theinexperienced students, those who have never completed an online evaluation, who prefer tocomplete the evaluation on their own time. Students with even one experience move toward theaverage; 84% prefer to complete evaluations on their own time. A higher percentage ofundergraduate students (86.1%) than graduate students (82.9%) favored completing evaluations
on their own time rather than as a class in a computer lab. It appears that with increased
familiarity, students do not need the support of going to the lab together, although some still
prefer this as they feel they will forget to do the evaluation on their own otherwise, as shown by
this comment from an experienced graduate male student:
It is very difficult for me to motivate/remind myself to complete online course
evaluations. I understand the effort to move towards a paperless process to savemoney and time, but it is much easier for me to complete the traditional paper
evaluations in class.
In addition, 22% of undergraduate students who favor traditionally delivery ofevaluations indicate the availability of help is one reason they prefer the traditional method.
Graduate females also indicated the availability of help was a reason they prefer traditional
evaluations. While experience with online evaluations tends to lead students to choose tocomplete them on their own, a sizeable minority of students experienced with online evaluations,and the more inexperienced students prefer to complete online evaluations in a computer lab orcomplete them in the traditional way in class, rather than on their own.
In terms of gender, some minor gender differences were revealed by the research.
Undergraduate males have more opportunity (53%) and experience (44% report taking anevaluation online five or more times) than undergraduate females (43% had opportunity and 40%
report taking online evaluations five or more times). Possible explanations include more males insuch fields as science, math, and computers, and a greater use of online evaluations in thesekinds of classes. It is also possible that male undergraduate students are more comfortable intaking online evaluations than females or graduate students when offered the opportunity.
Graduate students results were much closer, revealing few differences based on gender between
experienced students, and only a slight difference between inexperienced graduate students. Thismay be due to the fact the male and female students are taking the same courses as they are notseparated by major at the graduate level.
There were other differences in student responses indicating why one delivery method
was preferred over another. Female students, both graduate and undergraduate, indicate theanonymity and privacy of online evaluations were reasons they prefer them. Female students
chose these reasons 10% more than male students. Graduate male students who preferred online
173

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

evaluations choose the reason “Doesn’t Take Class Time” more than any other group (56.9% ascompared to the average of 48.2%), showing this as a unique concern for this group. Some
differences were apparent in the responses of students who preferred the traditional format fordelivery of evaluations. Female students were far more concerned with the difficulty of gaining
online access to the evaluations. Overall, 35.1% of female responses chose this as a reason for
preferring traditional delivery as compared to 6.2% of males. No undergraduate males choose“Hard to get online access” as a reason and only 8.3% of graduate males choose this reason.
All levels, undergraduate and graduate, and both male and female students, were unitedin choosing the convenience of online evaluations as the main reason they prefer this format. For
those who preferred the traditional format, “More likely to complete” was similarly the top
choice (94.6%) for all levels and genders.
In summary, format preferences and reasons are somewhat related to program level,
gender, and prior online evaluation experience. The most pronounced differences in responses
occur at the program level, with 92.5% of undergraduates favoring online evaluations ascompared to 83.8% of graduate students. Gender did not affect choice of format as much as otherfactors. Prior experience with online evaluations did affect the responses, as those with noexperience with online evaluations were more likely to choose the traditional method, andstudents with more experience with online evaluations preferred online evaluations more.
A close analysis of student responses and comments help to answer the third researchquestion, “What can we learn about improving student’s experiences with faculty evaluationsdelivered online?”
Graduate professors appear to offer less opportunity for students to complete online
evaluations, as 6% of graduate respondents have never had an opportunity to complete a facultyevaluation online. Possible reasons may be that more adjuncts teach graduate classes ascompared to undergraduate classes, and are not familiar with delivering evaluations online.
Graduate students may be perceived as less technologically able by their professors, and thus
unlikely to complete online evaluations, so they choose the traditional delivery method. Graduate
classes more often take place on weekends and evenings when computer labs (in which to takethe evaluations) are not available or perceived as not available. Graduate classes, far more thanundergraduate classes, are offered off-campus, and computer labs may well be locked or nonexistent in some of the locations where graduate classes are held. It is also possible professors
who teach graduate classes have characteristics which cause them to stay with the traditionaldelivery method rather than offering the evaluation online, but whatever the reason, College of
Education graduate students are offered fewer opportunities and have less experience with online
evaluations as compared to undergraduate students.
Undergraduate students have opportunities to take online evaluations from professors inother departments or colleges, while graduate students usually take all their courses from theCollege of Education. Undergraduates are exposed to a greater variety of professors, and course
experiences. For example, at the research site, there are no large lecture classes offered in theCollege of Education at either the undergraduate or graduate level. However, general education
classes such as math and history classes are taught as large lecture classes and these classes often
offer the faculty evaluation online, which increases undergraduates experience with online
evaluations. The survey indicated the level of experience with online evaluations influences
student preferences. Students who have completed more online evaluations prefer the online
format at a higher rate (see Table 8). Graduate students have less opportunity, thus lessexperience, and are less likely to prefer online evaluations than undergraduates.
174

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

The college needs to make an effort to overcome these differences in experience andencourage graduate professors to offer evaluations online. Possible solutions to closing the gap
involve better communication with off-campus locations to ensure a computer lab is available forthe last class, training for adjuncts as to how online evaluations work, and further research todetermine why graduate professors choose traditional delivery over online delivery ofevaluations.
Graduate students need more support the first few times they complete online
evaluations, and they need assurance their responses are anonymous. An explanation of theprocess (that the instructor cannot access the Blackboard evaluation course to see individual
responses) is needed for graduate students. Graduate students also have more concerns about
access to the evaluations, and may be better candidates for a whole class lab experience.
Graduate students may not use Blackboard as often as undergraduate students, and professors
can help by showing them how to access their password to avoid situations such as this one:
A lot of people didn't know their passwords in order to get online to complete theevaluations. This was the case for me 1/2 times. I had my password at home, but couldn't
remember it when we tried to do the evaluation in class.
Graduate students tend to be older and may have unique barriers such as the one
described by a graduate student in this comment, “The font on the online evaluation was so
small, it was unreadable in some of the questions.” Professors who teach graduate classes needto address students concerns and offer them encouragement to offer online evaluations, so that
more graduate classes begin to offer online evaluations.
Graduate students, particularly female graduate students, need to understand the only
thing different about online and traditionally delivered faculty evaluations is the delivery format.
They need to know that the survey is exactly the same otherwise, with the same questions andthe same space for comments. They also need to know that student comments are typed to ensure
anonymity before they are given to faculty. Student comments such as the following show
students do not realize that the evaluations delivered online and traditionally are exactly the same
except for the delivery method:(a) I think that the traditional evaluations always offer you a place to write toabout the professor or class and some on-line ones do not.
(b) Traditional bubble sheets provide a more personal level to suggest ways toimprove the course and/or compliment a teacher's style.
(c) I can fill out more specific qualities of my professors instead of having"cookie cutter" responses.
(d) Typical bubble sheet evaluations are pretty general in the questions that you
write responses to. I don't imagine that profs get that helpful of feedback. I thinkthat online evals provide the opportunity for more OR more specificquestions/feedback because people completing them have the opportunity to typeresponses.
(e) I feel the evaluations have a better chance of reaching those that need to readthem.
It appears to take only one to two experiences for the student to feel confident enough toprefer to take the online evaluation on his or her own time. It is a good idea for a professor topoll his or her students, especially a predominantly female graduate class, for familiarity with
online evaluations. If some of the students have never completed the evaluation online, it would
175

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

be helpful for the professor to reserve a computer lab the last night of class, and take those whowish to go to the lab as a group, and help them complete the login and other steps needed tocomplete the online evaluation. If a choice is available, the professor should chose a lab in which
students cannot readily see one another’s screens, or one with enough computers so that students
do not have to sit side by side, as some students did not like the idea that other students could see
their computer screens when filling out the evaluation. This could be the best option for students,
as it would solve the problem of students who do not know how to complete the evaluation
without help, while also allowing those who choose to return home rather than go to the lab tocomplete the evaluation on their own. It also may reduce the problem of students forgetting tocomplete the evaluation.
Students need to understand the importance placed on faculty evaluations and a timeframe established for completion that meets student needs. Students need to know what
professors control and what they do not, as illustrated in this student comment:
Profs need more training on how to set up course evaluations online. Many times
I have been rushed to submit the evaluation by the deadline because it was notmade available on time because the prof didn't know how to set it up.
At the research site, course evaluations are set up by the Blackboard administrator, not individual
professors.
Often evaluations are made available for the same time period for an entire campus or
department, and an individual professor has little say in when the online evaluation will beavailable to students. In other cases, professors can choose when the evaluation is madeavailable on an individual basis. Student comments follow:(a) Course evaluations should be done at the end of the last class day. I've hadclasses that it was completed at the beginning of the day and wished I could haveadded comments pertaining to that class session. I've also had evaluations thatneeded to be completed BEFORE the end of the last class day. The deadline for
evaluations needs to be after the class is completed, not before.
(b) I don't like have a short window of opportunity to fill them out. For one class
I forgot to do it and was not able to get back into the system to complete it.
(c) Needs to be more time available for turning them in online.
The above student comments indicate student dissatisfaction with the timing ofevaluations. It may be helpful for the school or department to decide on the availability andtiming of evaluations as a group to standardize the dates the evaluations are available. Students
need to be informed when the evaluations will be accessible (perhaps by email) and if this time
period is always the same students will know what to expect each semester. The online
evaluation of faculty can be a successful experience for more students if simple steps are taken toincrease student’s understanding of the processes involved and if student comments about theprocess are solicited and addressed in an ongoing manner. A possible solution is to have a spaceon the evaluations for a comment about the evaluation process, so students can continue to give
input in this area.
Conclusion
In conclusion, students overwhelmingly prefer completing faculty evaluations online.
The most significant reason students give for not preferring the online format was that they weremore likely to complete the evaluation when given to them in class. Other barriers to preferring
online evaluations, such as greater familiarity with the traditional format, lack of online access,176

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

and inability to ask for help, can all be easily overcome with a small amount of practice andexperience. The more experience students have with online evaluations the more they prefer thisformat, leading to more students over time preferring to complete online evaluations of faculty
online.
It is noteworthy that no student indicated he or she prefers online evaluations because theonline format made it easier to avoid completing the survey. On the contrary, many students
wrote that they felt they could give richer, more thoughtful, and more useful feedback when they
completed the online evaluation on their own, at a time of their choosing, with no time
constraints. If our students are willing to expend more effort to help us do our job better, it makes
sense to offer them every opportunity to complete student evaluations of faculty online.

177

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

References
Anderson, H. M., Cain, J., & Bird, E. (2005). Online course evaluations: Review of literature anda pilot study. American Journal of Pharmaceutical Education, 69(1), 34-43.
Carini, R., Hayek, J., Kuh, G., Kennedy, J., & Ouimet, J. (2003). College student responses to
web and paper surveys: Does mode matter? Research in Higher Education, 44(1), 1-19.
Dommeyer, C, Baum P., Hanna, R., & Chapman, K. (2004). Gathering faculty teachingevaluations by in-class and online surveys: Their effects on response rates andevaluations. Assessment and Evaluation in Higher Education, 29(5), 611-623.
Dommeyer, C, Baum, P., Chapman, K., & Hanna, R. (2002). Attitudes of business faculty
toward two methods of collecting teacher evaluations: Paper vs. online. Assessment and
Evaluation in Higher Education, 27(5), 455-462.
Donovan, J., Mader, C., & Shinsky, J. (2006). Constructive student feedback: Online vs.
traditional course evaluations. Journal of Interactive Online Learning, 5(3), 283-296.
Kuhtman, M. (2004). Review of online student ratings of instruction. College and University
Journal, 80(1), 64-67.
Laubsch, P. (2006). Online and in-person evaluations: A literature review and exploratorycomparison. MERLOT Journal of Online Learning and Teaching, 2(2), 62-73.
Layne, B., DeCristoforo, J., & McGinty, D. (1999). Electronic versus traditional student ratingsof instructions. Research in Higher Education, 40(2), 221- 232.
Ravelli, B. (2000). Anonymous online teaching assessments: Preliminary findings. Paper
presented at Annual National Conference of the American Association for Higher
Education, Charlotte, North Carolina.
Sax, L. J., Gilmartin, S. K., & Bryant, A. N. (2003). Assessing response rates and nonresponsebias in web and paper surveys. Research in Higher Education, 44, 409-432.
Thorpe, S. W. (2002, June). Online student evaluation of instruction: An investigation of nonresponse bias. Paper presented at the 42nd Annual Forum for the Association for
Institutional Research, Toronto, Ontario, Canada.
Wachtel, H. K. (1998). Student evaluation of college teaching effectiveness: A brief review.
Assessment and Evaluation in Higher Education, 23(2), 191-211.

178

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

AppendixStudent Survey: Course Evaluation Preference
1. Student Status:
Undergraduate
Graduate
2. Student Gender:
Male
Female3. Questions:a. Opportunities: Approximately how many classes have offered you the opportunity tocomplete the evaluation online?
0 classes
1-2 classes
3-4 classes
5 or more classesb. Completions: How many times did you actually complete an evaluation online?
0 times
1-2 times
3-4 times
5 or more timesc. Location Preference: Where would you prefer to complete online course evaluations?
I prefer to go to the computer lab with the whole class to complete them.
I prefer to complete them on my own timed. Format Preference: Given the choice between online evaluations and traditional bubble
sheet evaluations, which do you prefer?
I prefer the online format.
I prefer the traditional bubble sheet format.
e. Online Preference: If you prefer ONLINE course evaluations, check any of the following
reasons. (Check as many as apply--but only if you prefer ONLINE evaluations.)
I like the privacy of online evaluations.
I like having more time to think before I write.
Online evaluations don't waste class time.
I like the anonymity (no handwriting).
I can write more comments when I type.
I like completing it at my own convenience.
Other (please specify)

179

Journal of Interactive Online Learning

Donovan, Mader, Shinsky

f. If you feel satisfied with your answers above, please check "Go to End of Survey" andclick Next.
g. Bubble Sheet Preference: If you prefer TRADITIONAL bubble sheet course evaluations,check any of the following you agree with. (Check as many as apply--but only if you prefer
TRADITIONAL evaluations.)
It's sometimes hard to get online access.
I'm more familiar with bubble sheet format than online format.
I am more likely to complete it since we're right in class.
I can ask for help if I don't understand something.
Other (please specify)
h. You have successfully completed this survey. We appreciate your help with the study.
If you have any other comments, please provide them below

180