論文 / 著書情報
Article / Book Information
Title

Reminder effectiveness on two web questionnaire surveys for students

Authors

Tetsuya Oishi, Noriko Kuwano, Eiichi Takata, Masao Mori

CitationInternational Journal of Institutional Research and Management, Vol. 1,
No. 2, pp. 1-16

Pub. date2017, 9Powered by T2R2 (Science Tokyo Research Repository)

International Journal of Institutional Research and Management
International Institute of Applied Informatics
2017, Vol.1, No.2, P.1 - 16

Reminder effectiveness on two web questionnaire surveys
for students
Tetsuya Oishi ∗ , Noriko Kuwano † , Eiichi Takata ‡ , Masao Mori §Abstract
In Kyushu University, we had held web questionnaire surveys twice to obtain some educational information of the students. We reminded the students to answer the questions
certainly during the term of each questionnaire survey. The students are reminded to answer the questions not only by us but also by the faculty. As a result, the response rate of
the first questionnaire survey was 36.5%. Moreover, the rate of the second time was 40.6%.
In this paper, we explain our method of these questionnaire surveys and the reason why the
response rate was high.
Keywords: Institutional Research, Web Questionnaire Survey

1 Introduction
To plan ahead, decide something, or improve itself, Institutional Research is one of themost important activities [1]. In Japan, the national universities reports their activities tothe the government. Educational effects, outcomes of research, several items related to each
university, and so on are included in these reports. The experts evaluate every university
by reading these reports. The universities which obtained good evaluation are regarded as
good institutions in Japan or all over the world.
These reports usually include the facts which are better than other universities. The
staffs belonging to university have to obtain the evidences to demonstrate some effects or
outcomes to show some advantages. In Kyushu University, to self-check and to store somedata for showing faculty outcomes, the Researcher Database system is working. However,
we had no systems to obtain data from students.
In these conditions, we decided to introduce a web questionnaire system which will
be mentioned in Section 3 to obtain students’ information. We could know many factsof students by using this system: for example, students’ study hours, frequency of thesyllabus usage, and other activities. The students had to answer the all questions of thisquestionnaire survey. However, the students were imposed no penalties even if they didn’t
complete to answer the questions. Moreover, the students who had completed to answer the∗†‡§

Tokyo Institute of Technology, Tokyo, Japan
Kyushu University, Fukuoka, Japan
Kobe University, Hyogo, JapanTokyo Institute of Technology, Tokyo, Japan

2

T. Oishi et al.

questions gained no rewards. Generally, the method without rewards obtains few answers.
An experimental study about a survey response rate [2] reported that the response rate was
17.8% for no rewards answerers. In this study, authors also reported that the response rate
was 36.7% for those who were given some rewards. In other words, the response rate canbe influenced by whether there are some rewards or not.
Surprisingly, in our first questionnaire survey, we gained 36.5% (997/2,730) as response
rate with no rewards and no penalties. Moreover, in our second questionnaire survey, theresponse rate was 40.6% (2,385/5,869). In these questionnaire surveys, we reminded thestudents several times to answer the questions. The faculty also worked to remind them toanswer. It would be difficult to remind them to answer the questions if we had held thesequestionnaire surveys by paper. But it was very easy to remind them because we used a
web system. By using a web system, we could send several emails easily to students toremind to answer the questions. After reminding the students to answer, the number ofcompletion to answer the questions was increased exactly. In this paper, we show how we
obtain completed answers using reminders.
Web questionnaire surveys have several characteristics. Ohsumi et al classified web
questionnaire surveys at a viewpoint of answerers’ characteristics [3] [4]. There are three
types of classification as follows:
panel type All registered people can answer.
open type An indefinite number of people can answer.
resource type Selected targets from all registered people can answer.
Here, we pay attention to the resource type. The answerers of this type are selected after
registered beforehand. The resource type is divided into three methods as follows.
open method in the resource The questionnaire is conducted by applying open type forall registered people.
narrowed attribute method The targets satisfied certain conditions can answer.
sampling method in the resource Answerers are selected at random from the registeredpeople.
In Kyushu University, the faculty cooperated with our questionnaire survey. The students
as answerers belonged in the departments the faculty belonged. Therefore we think that ourquestionnaire survey is close to resource type. Moreover, sampling method in the resourceis the most similar to our questionnaire survey.
We had already described simply our web questionnaire survey held in 2013 [5] [6]. In
these papers, we showed only a fact obtained from this questionnaire survey. In this paper,
we additionally show not only the backgrounds but also the related works of our research.
We also show the effect of reminders and a reliability of it. Moreover, we compare between
two questionnaire surveys which were held in 2013 and 2015 respectively.
Then we show paper and web questionnaire survey in Section 2 and a Web questionnairesystem in Section 3. In Section 4, we show how to remind the answerers by using web
questionnaire. Section 5 shows the effects of reminders and we conclude in Section 6.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

Reminder effectiveness on two web questionnaire surveys for students

2 Paper and Web Questionnaire Survey
In this section, we explain a paper questionnaire survey before explaining a web questionnaire survey. There are some problems when conducting the paper questionnaire survey.
These problems are shown by giving some examples. Next, we show some examples of
the web questionnaire survey. Finally, we show the comparison between paper and web
questionnaire surveys held in Kyushu University.

2.1 Example of Paper Questionnaire Survey
We show three examples of the paper questionnaires surveys: Hokkaido University, Mobility Management, and Hiroshima University. We show these examples with these problems.

2.1.1 Hokkaido University
A paper questionnaire survey was conducted in Hokkaido University in 2006 [7]. The
targets of this questionnaire survey were students and faculty. The purpose of this questionnaire survey for students was to know the thoughts about the curriculum of this university,and one for faculty was to obtain the opinions about the education.
There were 15 questions for students, and 9 questions for faculty. The response rateof the questionnaire survey for students was 12.3% (333/2,717), and one for faculty was
48.6% (261/537). The students received the questionnaire paper when registering theircourse. However, the response rate was not high.
In this questionnaire, the students could earn nothing after completing to answer thequestions. They were also not reminded to answer the questions. We thought that most of
students had lost their motivations to contribute the questionnaire survey.

2.1.2 Mobility Management
In general, those who are invited to a questionnaire survey don’t always answer questions.
For example, the invited people may forget to answer the questions or may not notice thisinvitation. Therefore, to remind them to answer the questions is very important.
The Total Design Method (TDM) [8] is described by Dillman. TDM is one of methods to raise the response rate of questionnaire survey. The characteristics of TDM are as
follows:• To prepare regular patterns of invitation, reminder, and so forth.
• To send information to remind answerers to answer the questions.
• To decrease the number of questions.
TDM is very efficient method if the targets are clear.
A paper questionnaire survey about Mobility Management (MM) was conducted by
Hagihara et al [2]. MM is one of the measures to solve several traffic problems. They usedreminding method like TDM [8]. TDM is one of the methods to raise the response rate.
They used not only reminders but also rewards to raise the response rate. Needless to say,the reminders and the rewards both had cost many expense.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

3

4

T. Oishi et al.

2.1.3 Hiroshima University
In Hiroshima University, to evaluate their classes from 2004 to 2008, several questionnaire
surveys were conducted by using mark sheet for the students [9]. These questionnaire
surveys were held in each semester. The average response rate was 72.7% (62,440/85,927).
The questionnaire mark sheets were handed over to the students at the end of the semester.
Therefore, a high response rate was realized.
While they could collect many answers, a lot of money was used to conduct the paperquestionnaire. The cost was about 35 thousand dollar which was equivalent to 3,519,546
yen in Japan. Especially, converting the mark sheet data to the electronic data was very
expensive.
The authors described that there were several weak points to conduct paper questionnaires. For example, the staff may lose the questionnaire papers which have been alreadyanswered. Moreover, the anonymousness of the answers isn’t kept.

2.2 Example of Web Questionnaire Survey
Several works related to web questionnaire survey had conducted. First, we show an example using LimeSurvey as we also used. Next, we describe some works about problems andsolutions of web questionnaire survey.

2.2.1 Questionnaire using LimeSurvey
In Ritsumeikan University, several paper questionnaire surveys had been conducted until
2007 [10]. The staffs and faculty in this university had known that web questionnaire surveys were very efficient from a point of view labor and expense. Then, they decided to
conduct a web questionnaire survey using LimeSurvey in 2009.
They also recognized not only advantages but also disadvantages when using web questionnaire survey system. There are major advantages as follows:• No transportation expense• Reduction of labor• Traceability
These advantages are very important to conduct web questionnaire surveys. Especially,
most valuable advantage is that expense and labor can be low. We can also trace the answerers because individual answers are stored in the web questionnaire survey system. Thus,
not only they but also we thought that web questionnaire survey systems were very usefulwhen conducting questionnaire surveys.
Next, there are major disadvantages as follows:• Digital divide• Low response rate
• System fault
How do the answerers who cannot use Internet answer the questions given from web questionnaire survey system? The authors worried about it at first. In general, the response rate

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

Reminder effectiveness on two web questionnaire surveys for students

of web questionnaire survey isn’t high. The system may stop suddenly according to circumstance. In recent years, those who cannot use Internet become less. Especially, in almost all
university, students can use Internet by using computers, mobiles, and so on. System fault
hardly occur. Therefore, the authors decided to use the web questionnaire survey system
LimeSurvey.
Email function has been implemented in LimeSurvey as mentioned in Section 3.5.
However, even though the email function was very useful, the authors didn’t use this function because of a danger to individual information. They sent emails to students who wereanswerers manually instead of the email function. They also sent postcards to remind themto answer the questions from their web questionnaire survey. The response rate became
27.9% (326/1,168) in the end. They thought that their questionnaire survey played well.
Though they would like to reduce expense and labor when conducting their questionnaire survey, they actually needed them a lot. Sending postcards to the students was very
effective. We thought that they needed many money to send postcards. They might behappy if they could have obtained high response rate with low expense.

2.2.2 Problems
There are several problems even though the web questionnaire surveys are very useful. In
this section, we explain several examples of the problems on web questionnaire surveys.
Kakiyama et al explained the quality of web questionnaire surveys [11]. They said that
it was difficult to understand whether the answerers could answer the questions exactly
even though web questionnaire surveys were not only easier but also cheaper than paperquestionnaire surveys. That is to say, an answer which an answerer should be answered
may not be the answer which he or she answered exactly. The other person can answer thequestions instead of the true answerer by disguising as him or her.
Murase explained the other problems of web questionnaire surveys [12]. In general,
there are various answerers in the world. In other words, answerers’ age, gender, educational background, and so on, are different each other. He said that a questionnaire result
obtained from the random targets might be incorrect. If YES-answered rate of a questionis 60%, it may become 20% or 90% really. Therefore, he also said that it was important toknow the answerers’ age, gender, educational background, and so on beforehand.
In Kyushu University, the answerers of our web questionnaire survey were the studentsof our university. The students as the answerers could answer themselves because we sent
them directly emails to introduce and remind the web questionnaire survey. As a matterof course, we had known their age, gender, and educational background. Therefore, theenvironment of web questionnaire in Kyushu University was properly.

2.2.3 Solutions
Many answerers don’t answer all questions or even don’t begin start answering in web
questionnaire survey. This is one of the problems on conducting web questionnaire survey.
The response rate can be raised if these answerers are reduced. There are several works tosolve this problem.
Yamamori et al developed a web questionnaire system. In this system, the administrator can recognize who don’t contribute the questionnaire survey [13]. The answerersof this system were anonymous. In general, there are several requests as follows when aquestionnaire survey is conducted.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

5

6

T. Oishi et al.

• High response rate
• One person can answer once
• Anonymousness
Their system satisfied these requests listed above.
To keep anonymousness, they used tokens. A token is a character string to distinguishindividuals. They sent an email to the answerers. This email contained individual token.
This token enabled every answerer to contribute the questionnaire survey only once. If an
answerer completed to answer all questions, the evidence of it exactly was stored in the web
questionnaire system. Therefore, they could recognize who had not competed to answer thequestions.
This system was developed by using sympoll that is one of PHP program. Multi questionnaire surveys could not conducted at the same time unfortunately. They could realize
this function if they could operate the database manually.
This solution becomes to be standard afterward. LimeSurvey which we used also provided these functions. LimeSurvey has enough functions to raise response rate up, to keep
anonymousness, to send email to remind to answer the questions, and so on. Especially,
reminding function is very useful. In next section, we explain the effect of reminders indetail.

2.3 Comparison between Paper and Web Questionnaire Survey
In Kyushu University, two questionnaire surveys had conducted to know the actual conditions of the students’ life in 2007 [14] and 2011 [15]. The former questionnaire survey was
conducted by paper and the answerers were selected by random sampling. The latter questionnaire survey was conducted by web. While the response rate of the former was 45.0%(2,299/5,107), the rate of the latter was 19.6% (3,195/16,342).
In general, the response rate of paper questionnaire survey is bigger than the rate of
web questionnaire survey. However, the number of answers was opposite. The targets ofthis web questionnaire survey are more than the targets of this paper questionnaire survey
distinctively. Then the number of answers for the web questionnaire survey also increased
as a matter of course.
As a result of this, we thought that the web questionnaire was very useful tool. In
addition, conducting the web questionnaire survey, we used LimeSurvey [16] which was
one of web questionnaire survey systems. It is interesting to see that 36.5% response rate
in 2013 and 40.6% in 2015 are significantly different from 19.6% response rate in 2007.

3 LimeSurvey
In general, paper questionnaire surveys need a lot of money and labor as described in previous section. Therefore, in this section we introduce web questionnaire surveys to solve
several problems occurred in paper questionnaire surveys. We explain LimeSurvey [16]which is one of web questionnaire survey systems.
LimeSurvey is one of open source web questionnaire survey systems. LimeSurvey isequipped with many sophisticated functions. We can use it free of charge because it is open
source. Therefore we can keep down an initial cost when starting a questionnaire survey. In

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

Reminder effectiveness on two web questionnaire surveys for students

Kyushu University, we needed no expenses even to prepare hardware because we had own
virtual environment. The version of LimeSurvey we used is “1.92+ Build 120919”.
LimeSurvey has main 6 functions.
1. System Management
2. Questionnaire Management
3. Design Template
4. Create Question
5. Email
6. Total and Analysis
The mail function is the most important among these functions. In following sections, we
mention the detail of these functions.

3.1 System Management
We can create users as questionnaire survey managers. These users are also given permissions to manage questionnaire surveys. These permissions are to execute, modify, delete,
and read questionnaire surveys. We can also create groups some users can belong to. Of
course, these groups also are given permissions.
These created users can log in LimeSurvey. They can utilize some functions after logging in. A special user who can do everything in LimeSurvey is given initially after installing LimeSurvey. When using LimeSurvey for the first time, you have to log in as the
special user.

3.2 Questionnaire Management
A user given the permission to manage questionnaires can change and configure questionnaires. For example, the user can the configure the date which a questionnaire survey should
start and end. The content of a configuration can be not only downloaded but also importedby the user. The user can register answerers to LimeSurvey. It is a very important roll. The
user can also manage questionnaires in detail.
Multi questionnaires can be managed by the user. The user can know the conditions ofevery questionnaires easily through the list of all questionnaires. In each questionnaire, theuser also can create, delete, and modify some questions. Questions can be rearranged. The
user can assume a question indispensable if the question is necessary to be answered. In
this way, the user can operate questionnaires and those questions variously.
The user also can replicate a questionnaire. When the user would like to make a questionnaire same as other questionnaire which have already conducted, the user can prepare
the new one by replicating other one. To replicate a questionnaire, the user only downloads
a configuration of other questionnaire and uploads it as a new questionnaire.

3.3 Design Template
A questionnaire page can be changed easily by selecting a design template. The user canedit, import, and export the design template. If the user have the knowledge of HTML andthe style sheet, editing a design template is very easy.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

7

8

T. Oishi et al.

3.4 Create Question
Questions are created by using various forms. Single-selective question such as radio buttonand pull-down menu is available if the user would like to get only one answer from aquestion. Multi-selective question such as checkboxes is used to get multi answers. Matrixselective question, text box to describe freely, and so forth are also available.
There are complex question forms. The user can use random order choices. The user
can create an additional question which is appeared if a condition is satisfied,. Of course,
the questions the user created can be previewed.
The user can replicate and reuse a question which has already created. If you use thisreplication method, you don’t have to create all questions individually when you would liketo create many questions.

3.5 EmailThe user can send the answerers some kinds of emails and the answerers also can receive
these emails. In this section, we describe intensively the email sent to the answerers.
First, the user can send the answerers an invitation email before starting a questionnaire
survey. This email contains some information as follows:• Explanation of the questionnaire survey• URL linked to the questionnaire site
• Email address for inquiry
A token is contained in the URL. Answerers are specified by the token. After accessing
the URL, the answerers can start answering all questions on this questionnaire survey. An
email which appreciates answerers’ contribution will sent to answerers after they complete
to answer the all questions.
An email for reminding to the answerers who didn’t complete to answer all questions
is also sent. The user can know who didn’t answer the questions completely. The user
only send such answerers an email to remind to answer the questions. A URL linked to thequestionnaire site is also contained in the reminding email. Therefore, the answerers canstart answering the questions immediately and easily after receiving it.
The user can prepare email templates. These emails are created based on them. The
names of the answerers can be included in these emails. The email function is very useful. This function contributed greatly to increase the number of answerers who completed
answer the questions greatly.
Moreover several emails which notice something important can be sent to the user. For
example, these emails include the noticing email sent to the user when the several emails tothe answerers failed to be sent.

3.6 Total and Analysis
The user can confirm the total of answers obtained from questionnaire surveys. Several
graphs created from the answers can be provided to the user. The user can operate the dataof answers by using CSV format files downloaded from LimeSurvey. These functions areavailable to analyze the answers. The user can decide and find something significant byanalyzing answers.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

Reminder effectiveness on two web questionnaire surveys for students

Many statisticians usually use “R” which is one of statistical computing environments.
R is free software programming languages. The file suitable for R can be created fromLimeSurvey. You can analyze the answers more statistically by using R.

3.7 Others
LimeSurvey has many functions as we mentioned above. In this section, we show the otherimportant functions.
The answerers can know how many questions have left during answering the questions.
LimeSurvey has a progress bar which show the answerers the ratio of the number of questions which have already answered to one which haven’t answered. Therefore the answerers
feel easy.
All answerers can not only stop answering but also resume later. LimeSurvey has alsoa resumable function. Then they can restart answering at the point suspended.

4 How to remindReminding the answerers is very effective method when we would like to gather more answers on web questionnaire survey. In this section, we explain the comparison between
paper questionnaire survey and web questionnaire survey from a point of reminding answerers. We show that reminding answerers on web questionnaire survey is easy whilereminding answerers on paper questionnaire survey is troublesome.

4.1 Reminder in paper questionnaire
It is difficult to remind the answerers to answer questions on paper questionnaire survey.
Especially, if answerers can optionally decide whether they contribute it or not, reminding
is more difficult. In general, the answerers tend not to contribute it without rewards.
Hagihara et al showed the methods of questionnaires [2]. They conducted paper questionnaire surveys in three conditions as follows.
1. Handing over
2. Posting
3. Mailing
The cost they should pay was not free until the answerers could complete to answer allquestions provided on the questionnaire survey. They also reminded the answerers to answer the questions. Then, they also needed expense and labor to send the answerers several
mails to remind.
Furthermore, they examined how rewarding influenced the response rate. Naturally, theresponse rate of those who were given rewards was higher than those who were given
nothing. The influence of reminders and rewards is shown in Table 1. While the response
rate was 21.3%(17/80) without reminders and rewards, the one was 57.5%(46/80) withthem. In this way, reminders and rewards influenced response rate of questionnaire survey.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

910

T. Oishi et al.

Table 1: The relation between the reminders and the rewards
Conditions
Without rewards With rewards
Without reminders
21.3%(17/80)
37.5%(30/80)
With reminders
28.8%(23/80)
57.5%(46/80)

4.2 Reminder in web questionnaire
In this section, we explain the reminders on web questionnaire survey. First, we explain an
example of Hiroshima University. Then, we also explain our work in Kyushu University indetail.

4.2.1 ENGSYS and MOMIJI (Hiroshima University)
The paper questionnaire survey conducted in Hiroshima University has already shown in
Section 2.1.3. This university also conducted a web questionnaire survey from 2003 [9].
In this section, we describe the outline of the web questionnaire survey and the method toremind students as answerers to answer the questions.
In Hiroshima University, ENGSYS which was developed in this university was usedfrom 2003 to 2008. Moreover, the university used MOMIJI which was also developed inthis university from 2009. ENGSYS and MOMIJI are total management systems for thisuniversity. These systems has the function of questionnaire survey. A questionnaire survey
was conducted to evaluate classes of the students in this university.
The web questionnaire survey had been conducted in the term of exam. The response
rate should have been high because almost all student should go to the university in thisterm. Reminders using emails were sent to the students. However, the final response rate
was at most 32.2%. Then, at next time, computer rooms were prepared and the students
were let to use this room to contribute the questionnaire survey. Unfortunately, the response
rate was almost same as the last web questionnaire survey.
To raise the response rate, they decided to impose penalties on the students if they
didn’t complete to answer the questions from the web questionnaire survey. As a result, theresponse rate became to be more than 80%. The student who didn’t complete to answer
the questions had to explain the reason not to complete to answer as a penalty. Needlessto say, the penalty influenced the response rate. Of course, the reminders were also sent to
students.
MOMIJI was started using in 2009. The response rate was 48.5% because the reminderfunction was not used on the same year. The penalties were also imposed on the students
who didn’t complete to answer the questions. In 2010, the response rate was raised up to
76.0% because the reminder function was started using again. Therefore, the reminders
were very efficient to raise the response rate up obviously.
The staffs of Hiroshima University did a troublesome task beforehand while the reminders they had sent worked very well. Actually, they had to the data of answers tospecify the students who didn’t complete to answer the questions. That is to say, it was noteasy to send emails to such students.
We can know who didn’t complete to answer the questions if LimeSurvey was introduced. Moreover, after we just click a few times, we can send emails to them easily. In thenext section, we explain our web questionnaire survey using LimeSurvey.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

Reminder effectiveness on two web questionnaire surveys for students

4.2.2 LimeSurvey (Kyushu University)
To obtain students’ information, we adopted a web questionnaire survey system which is
LimeSurvey. Using the web questionnaire survey system, the user as an administrator ofthis system can remind the answerers to answer the questions of the questionnaire survey
by sending emails. The user just clicks the button to remind answerers. Emails to remind
are sent to the answerers after this action. The answerers confirm that they must to answer
the questions from the questionnaire survey after receiving the email.
It is not necessary to prepare several papers of the questionnaire survey. Moreover, wedon’t need to hand over, post, and mail. No expense is needed to remind. It is also easythat we specify the answerer who should be sent emails to remind because LimeSurvey has
these information.
LimeSurvey provides other useful information. We can know how many emails wehave to send to the answerers who didn’t complete to answer the questions. Therefore
we can know effectiveness of the reminders easily because we get this information. For
example, it becomes clear that several answerers didn’t read emails if they didn’t access the
web questionnaire survey system even though the reminder emails were sent many times.
In Kyushu University, we didn’t fail to send emails at first questionnaire survey because
all students have their own email address given by our university when they entered ouruniversity.
In our first questionnaire survey held in 2013, the response rate was 13.6% before wereminded at first. Finally, the response rate became 36.5% after reminding three time.
Moreover, in our second questionnaire survey held in 2015, the response rate was 15.1%
before reminded at first, and final one became 40.6%.

5 Effect of reminding
In this section, we explain our questionnaire surveys held in Kyushu University at 2013 and2015, and their effects of reminding.

5.1 Abstract of our questionnaire
We have held two questionnaire surveys. The first questionnaire survey was held in 2013,
and the second is held in 2015. The targets of these questionnaire surveys were the students
who belonged to Kyushu University. Almost all these students are final grade students
in the graduate and undergraduate course. The number of students intended to contributeto the first questionnaire survey was 2,730, and one for second questionnaire survey was
5,869. We didn’t give any rewards to the students for completing to answer the questions.
We also didn’t impose any penalties on students even if they didn’t complete to answer
the questions. Moreover it was optional for students whether they would contribute to ourquestionnaire surveys. However, we reminded them three time to answer the questions asfollowing date because we needed many answers from students.
• First questionnaire survey– November 5, 2013
– November 14, 2013
– November 20, 2013

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

11

12

T. Oishi et al.

Table 2: The number of the students who completed to answer in each period
First Questionnaire Survey (2013)
Subtotal
Cumulative Total
1st period 371 (13.6%)
371 (13.6%)
2nd period 396 (14.5%)
767 (28.1%)
3rd period 140(5.1%)
907 (33.2%)
4th period
92(3.3%)
997 (36.5%)
Second Questionnaire Survey (2015)
Subtotal
Cumulative Total
1st period 889 (15.1%)
889 (15.1%)
2nd period 841 (14.3%) 1,730 (29.5%)
3rd period 383(6.5%) 2,113 (36.0%)(4.6%) 2,385 (40.6%)
4th period 272• Second questionnaire survey– November 2, 2015
– November 10, 2015
– November 17, 2015
In addition to the reminders, the faculty as academic advisor for the students reminded themat the same time. We could give the faculty the list of the students who didn’t complete toanswer because LimeSurvey could provide us with this list easily.

5.2 EffectWe divided the period to contribute to the questionnaire surveys into 4 periods as follows.
• First questionnaire survey– 1st period: From October 11 to November 4, 2013
– 2nd period: From November 5 to November 13, 2013
– 3rd period: From November 14 to November 19, 2013
– 4th period: From November 20 on• Second questionnaire survey– 1st period: From October 26 to November 1, 2015
– 2nd period: From November 2 to November 9, 2015
– 3rd period: From November 10 to November 16, 2015
– 4th period: From November 17 on
Table 2 shows the number of students who completed to answer the questions in eachperiod. Fig. 1 and Fig. 2 show the daily variation of the number of students who completed
to answer at 2013 and 2015 respectively. In these graphs, the horizontal axis mean thedates and the vertical axis means the number of students who completed to answer at each

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

Reminder effectiveness on two web questionnaire surveys for students

Figure 1: The number of the answers per day (First time)date. Four peaks are recognized in these figures respectively. The first peak is found on
the first day of these questionnaire surveys. The other peaks are found on the dates wesend the students reminder emails to contribute our questionnaire surveys. We explain theeffectiveness of the reminders on our web questionnaire surveys in the following part.
At the first dates of these questionnaire surveys started, 112 and 580 students had completed to answer the questions at 2013 and 2015 respectively. But the daily numbers of thestudents completing to answer were decreasing until the date we reminded the students tocontribute our questionnaire survey.
Then, we reminded the students at first on November 5, 2013 and November 2, 2015
respectively to increase the number of the students who completed to answer the questions.
As a result, 185 and 615 students had completed to answer the questions respectively at thedate the reminder emails were sent. These numbers of students completing to answer per
day was the highest value. These results shows that the reminders were very effective toincrease the number of students to contribute our web questionnaire surveys.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

13

14

T. Oishi et al.

Figure 2: The number of the answers per day (Second time)

Until the questionnaire surveys ended, we sent reminder emails again to raise the response rates up more and more. The second reminder emails were sent on November 14,
2013 and November 10, 2015. The number of students completing answer were 86 and 261
which were also high value as we expected.
The last reminders were sent on November 20 2013 and November 17 2015 respectively. Fig. 1 and Fig. 2 also show that these reminders were effective.
The last response rates were 36.5%(997/2,730) and 40.6%(2,385/5,869) respectively.
The response rate might raise up to 20.0% at the most if the reminder emails were not sent
to the students. Therefore the reminders were very effective.
The response rate of second questionnaire survey greater than one of first questionnaire
survey. The contents and the methods to conduct these questionnaire surveys were almost
same. In second questionnaire survey, the email addresses we sent several emails were theones the students usually used. Therefore, many students could read the emails sent fromLimeSurvey exactly. This contributed to raise the response rate up.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

Reminder effectiveness on two web questionnaire surveys for students

6 Conclusion
In this paper, we explained the effectiveness of the reminders by using emails in our web
questionnaire surveys. We also compared two questionnaire surveys we had held on 2013
and 2015 respectively. We introduced LimeSurvey as a web questionnaire survey system.
We could send the reminder emails easily to the students to be contributed our questionnaire
surveys. The faculty of these students who are targets of our questionnaire surveys also
worked to remind them to contribute. Actually, we could raise the response rate up by using
the reminder function. The response rate were 36.5%(997/2,730) and 40.6%(2,385/5,869)
on 2013 and 2015 respectively even though the students could contributed to our questionnaire surveys optionally without any rewards or penalties. As the response rate of general
web questionnaire survey was at most 10%, we can say that these response rates were reallyhigh.
In future work, we have to survey the other difference between these two questionnaire
surveys. For example we didn’t take the attributes of student into consideration on thisresearch mentioned above. The students who have specific attributes may answer quickly.
Moreover, the students who contributed to our questionnaire surveys at the same time may
have same attributes.
Actually, there were many students who didn’t contribute our questionnaire surveys.
Some of them maybe could not read the invitation and the reminder emails. Other of themmaybe ignore the questionnaire surveys even though they could read these emails. We will
work hard that the response rate will be raise up using the web questionnaire survey systemafter confirming whether our method was really appropriate or not.

Acknowledgments
This work was supported by JSPS KAKENHI Grant Number 15K04305.

References[1] J.L. Saupe, “Functions of Insitutional Research, 2nd Edition”, published by Association
for Institutional Research, 1990.
[2] Go Hagihara, Hiroyuki Ota, and Satoshi Fujii, “An experimental study about the survey response rate”, Basic study of efficient strategies to raise the participation rate of
Mobility Management, 2006.
[3] Noboru Ohsumi, “Internet Surveys: A Review of Several Experimental Results - Applying Data Science Approach to the Exploration of Internet Survey -”, The Behaviormetric Society of Japan, Vol.29, No.1, pp. 20-44, 2002[4] Osamu Yoshimura and Noboru Ohsumi, “Quality of Internet Survey”, ISM Symposium, pp. 15-32, 2003
[5] Tetsuya Oishi, “Effects of reminders in questionnaire survey”, Proceedings of the 3rdInternational Conference of Institutional Research and Institutional Management 2014,
Sep. 2014.

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.

15

16

T. Oishi et al.

[6] Masao Mori, Tetsuya Oishi, and Eiichi Takata, “Effects of reminders in web questionnaire survey for students”, Proceedings of the 30th annual conference of Japan societyof educational information, Oct. 2014.
[7] Toshiyuki Hosokawa, Toshiyuki Nishimori, and Atsushi Ando, “Results of Questionnaire about New Curriculum for Freshmen in 2006”, J. Higher Education and Lifelog
Learning 15, pp99-112, 2007.
[8] Don A. Dillman, “Mail and telephone survey: the total design method”, 1978
[9] Satoshi Taoka and Toshimasa Watanabe, “A Web-Based Questionnaire and Its Evaluation of Class-Assessment-by-Students”, The Transactions of the Institute of Electronics, Information and Communication Engineers D, Vol J97-D, No.5, pp. 1024-1034,
2014.
[10] Koichi Higuchi and Miki Nakai, “The practice of web surveys using free software”,
Ritsumeikan Social Science Review, Vol.45, No.3, pp.69-82, Dec. 2009.
[11] Koichiro Kakiyama and Akira Harada, “Attempt Concerning Reliability Improvementof the Internet Questionnaire - Application of Mouse Tracking -”, Proceedings of the
52nd annual conference of JSSD, pp. 236-237, 2005[12] Yoichi Murase, “Electronic Survey”, http://www.asahi-net.or.jp/ BV7y-mrs/, 2006
(cited 2014-08-12)[13] Kazunori Yamamori and Risa Takagi, “Development of anonymous questionnaire system that can pursue unanswered person”, Departmental Bulletin Paper of Mie University, pp. 25-31, 2011[14] Kyushu University “Report of students’ life 2007”, Mar. 2008.
[15] Kyushu University “Report of students’ life 2011”, Aug. 2012.
[16] LimeSurvey, http://www.limesurvey.org/

Copyright © by IIAI. Unauthorized reproduction of this article is prohibited.