This is the peer reviewed version of the following article: Seki, Y. , Matsuura, K. , Sano, M. and Ueta, T. (2018), A questionnairesystem with an authentication framework considering online activities of users. IEEJ Trans Elec Electron Eng, 13: 1071-1072.
doi:10.1002/tee.22665, which has been published in final form at https://doi.org/10.1002/tee.22665. This article may be used for
non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.

IEEJ Transactions on xx
Vol.137 No.1 pp.1–2
DOI: 10.1541/ieejxxs.137.1Letter

A Questionnaire System with An Authentication Framework Considering
Online Activities of Users
Yosuke Seki∗
Member,∗Masahiko Sano Non-member,

Kenji Matsuura∗ Non-member
Tetsushi Ueta∗
Non-member

It is diﬃcult to distribute online questionnaire adequately to those subjects who satisfy requirements of investigations. In addition, low questionnaire response rates, because of factors such as overlooking request emails and noncooperation to answer the questionnaires, sometimes annoy the investigators. Now that today’s organizational systems
adopt a unified authentication framework of which logging function stores authentication records of users in federated
systems, the possible solution for these problems arises by utilizing the log data and adding the questionnaire process
to the framework. The present paper proposes a flow controlling mechanism arranged in a major unified authentication framework and describes how to utilize log data on the framework to select target persons for questionnaires andincrease the response rates.
Keywords: Questionnaires, Shibboleth, Questionnaire response rates

a questionnaire screen is displayed only to the target persons
after authentication. Our proposal presumes the situation inwhich the operators of the questionnaires (e.g. persons ininformation management centers) manage multiple systems
and investigate systems by the questionnaires.
2. The Proposal of The Questionnaire Method
2.1 The method to select the target persons
It
is considerable that the use states are grasped from (A)user identifiers(=who), (B) used systems(=what), (C) timestamps(=when), (D) client network addresses(=where) and(E) client user agents(=how). The systems record the information, such as uid and timestamps, to their log data. However, it is diﬃcult to apply common procedures for collecting
the information of various systems without central logging
services. Even if the central logging services are supplied toeach system, it is still diﬃcult to unify the data format and thecontent policy of the systems. Single-Sign-On (SSO) is oneof the best solution against such problems. Shibboleth [3]is one of the major framework of inter-organizational SSO,
and the present study adopts it. Shibboleth contains an identity provider (IdP), a service provider (SP) and a user agent(UA). The IdP plays a role in both a unified authentication
service and a central logging service for the authentication,and the SP provides services to the UA. It is possible to obtain (a) uids, (b) access destinations to the SP, (c) timestamps
and (d) IP addresses of access sources from the log data of
the IdP. The authentication screen of the IdP is displayed ona web browser of the UA, and the IdP generally uses Apache
which records (e) operating systems, browsers and other information to the log data.
A function is provided to select the target persons on thebasis of the use states. The operators combine any information from (B) to (E) in the function depending on their investigation purposes. (A) is used to extract the uids of the target
persons, according to requirements specified by the operators. (B) is used to select systems among federated systems

1. Introduction

In universities, students and faculties use various kinds ofsystems (e.g. an email system and a library system). Usability, visibility and other items are frequently investigatedfor various reasons such as replacement of the systems. Online questionnaires, distributed with emails directly or web
sites, are used widely for the investigations. Target persons
for questionnaires should be changed, depending on the investigation purposes. For example, when opinions and impressions of the systems are grasped, it is desirable that persons who have used them become the target persons. However, the use states of the systems are diﬀerent among persons, and consequently, it is necessary to judge each person
and distribute the questionnaires only to the target persons.
Furthermore, it seems that questionnaire response rates tend
to be low due to factors (e.g. overlooking request emails andnoncooperation to answer the questionnaires).
Some studies of the questionnaires to improve the response
rates are reported. Keding et al. [1] send short messages asreminders to the target persons. Their findings suggest thatthe short messages are eﬀective to improve the response rates.
Kitagawa et al. [2] develop a questionnaire system which cangrasp respondents and nonrespondents. Their experiments
show that the response rates improve. However, even if thetarget persons receive reminders or re-requests by the shortmessages and other methods, they may not answer the questionnaire because of the factors. In addition, both studies do
not propose methods to select the target persons.
The present paper proposes a method of the questionnaires
with a unified authentication framework to select the target
persons depending on the investigation purposes and to improve the response rates. In particular, (I) the target persons
are selected by utilizing log data of the framework, and (II)∗Tokushima University, 2-24, Shinkura-cho, Tokushima 7708501, JAPANc 2017 The Institute of Electrical Engineers of Japan.
⃝1

A Questionnaire System with An Authentication Framework Considering Online Activities of Users (Yosuke Seki et al.)by Shibboleth for deciding investigation targets. If persons
have used the systems selected by (B), they become the target
persons. (C) is used to select used periods (e.g. from April 1,2016 to March 31, 2017). (D) is used to select used locations.
Since the locations are judged by IP addresses, the operators
can select inside or outside of the universities. (E) is usedto select used devises (e.g. iPhone and a Windows PC). The
operators can select AND or OR condition to each item. For
example, if the operators specify “persons who have used theemail system and the library system with their iPad from outside of the universities in 2017” as the requirements, uid as
the target persons are selected for the questionnaires.
2.2 The method to display the questionnaire screen
To distribute the questionnaires in consideration of the factors
is needed to improve the response rates. If the questionnaire
screen is displayed not from the emails, without workload of
the target persons to obtain their cooperation, it may enhancepossibility of answering the questionnaires. The questionnaire screen is therefore displayed forcibly only to the target
persons after authentication. In other words, if the target persons access to the systems and their authentication results indicate a success, the questionnaire screen is displayed subsequently. The systems to display the questionnaire screen are
selected from (B). Since the questionnaire screen is displayed
forcibly, it may be diﬃcult to answer the questionnaires during class and work. To solve this problem, the questionnaire
screen includes a skip function.
2.3 System Development
A questionnaire system isbuilt with PHP, HTML and JavaScript. The questionnairesystem has functions of making the questionnaires, specifying requirements, distributing the questionnaires and obtaining the results of the questionnaires for the operators. The
usage states are collected from the log data of the IdP (idpprocess.log) and Apache (ssl request log). The questionnaire
process is added after the authentication process with FPSP[4], which is a plug-in software of the IdP for access controlof the users. FPSP is extended to redirect the access destination of the target persons to the questionnaire system. Figure
1 shows the sequence of the questionnaire method. The blue
arrows are original sequence of Shibboleth, and the green arrows are extended sequence. The IdP redirects the accessdestination of the UA to the SP after the authentication (iii)in the original sequence. In our system, the IdP confirms theUA whether the target persons or not (iv), and only the accessdestination of the target persons is redirected to the questionnaire system (v). In other words, by adding the questionnaire
process to Shibboleth, the questionnaire screen is displayed(vi) only to the target persons selected by requirements after the authentication. Afterwards, the questionnaire system
redirects the access destinations of the target persons to theSPs after the end of the questionnaires (vii), and the SPs provide their service to the target persons (viii).
3. A Trial Run of The Questionnaire System
Questionnaires are distributed to target persons by a conventional method and our method, and the response rates arecompared to verify eﬀectiveness of our method. One SP in
our university is selected as an investigation target. The coverage of the trial run is persons who have used the SP in2015. The target persons are 14 university students. They
are divided into two groups (a control group and an exper-63L LL 

4XHVWLRQQDLUH
V\VWHPYLLL YLL YL h'%

7DUJHWSHUVRQY LY LLL ,G3Fig. 1.

The sequence of the questionnaire methodimental group), considering usage frequency of the SP. The
period of the trial run is one week. The questionnaires aredistributed to the control group with emails as the conventional method and the experimental group with our system.
The students do not expect to answer the questionnaires because of no previous notice of the questionnaires. The trial
run yields the following results: the response rates were 0.14(the control group) and 0.71 (the experimental group). The
response rate of the experimental group was higher than thatof the control group, which suggests that there is possibilityof eﬀectiveness to improve the response rates with our system. However, one student commented that it had been alittle diﬃcult to answer the questionnaires during busy time.
It is considerable that to limit the questionnaire time, suchas only evening when the target persons may have time toanswer the questionnaires, is eﬀective.
4. Conclusion
The present paper proposes the method of questionnaires
with Shibboleth. The operators can specify requirements toselect the target persons by utilizing log data, and the questionnaire screen is displayed only to them after authentication. The results of the trial run suggest that the response ratesof our system are improved, compared to that of the conventional method. However, the trial is run in small scale. Accordingly, our system needs to be evaluated with large scale
systems which are used by many students and faculties. Inaddition, our system uses Shibboleth of version 2 which is
not latest one, and hence it is necessary to rebuild our systemwith latest version.
Acknowledgment
This work was supported by JSPS KAKENHI Grant Number JP15K12168.
References
(1)

(2)

(3)
(4)

2

Keding A, Brabyn S, MacPherson H, Richmond S.J, Torgerson D.J. Text message reminders to improve questionnaire response rates. Journal of Clinical
Epidemiology 2016; 79:90–95.
Kitagawa T, Oka H, Kaji Y. An Anonymous Questionnaire System for Rating Faculty Courses in Universities. Transactions of Information Processing
Society of Japan 2003; 44(9):2353–2362.
Shibboleth. https://shibboleth.net/. Accessed March 1, 2017.
FPSP. https://meatwiki.nii.ac.jp/confluence/pages/viewpage.action?pageId=12158554. Accessed March 1, 2017IEEJ Trans. XX, Vol.137, No.1, 2017