SUGISAKAMASANORI

Journal of Robotics, Networking and Artificial Life
Vol. 9(4); March (2023), pp. 369–373
ISSN (Online):2352-6386; ISSN (Print):2405-9021
https://alife-robotics.org/jrnal.html

Li-Min Chuang1, Hsiu-Hao Liu2
1

The Department of International Business, Chang Jung Christian University, No. 1, Changda Rd., Gueiren District, Tainan City, 711301, Taiwan
The Ph.D. Program in Business and Operations Management, College of Management, Chang Jung Christian University, No. 1, Changda Rd., Gueiren District, Tainan City, 711301, Taiwan
2Received 25 November 2021
Accepted 24 October 2022Service quality
Counseling service system
Case management
Modified delphi method
Analytic hierarchy process1.

The counseling service system is a case management tool used by professional guidance counselors and full-time teacher-counselor. However, even after multiple maintenances, system satisfaction and willingness to use it cannot be improved. This study used the modified Delphi method
and analytic hierarchy process to construct the service quality index of the counseling service
system by adding expert suggestions from different backgrounds. The study found that the service
quality indicators of the counseling service system can be divided into five criteria and 21 subcriteria to explore, among which were "perfect confidentiality mechanism", "web design & layout
meet the needs of the work", and "functional operation is simple and convenient" most valued byexperts and users. Based on the research results, this study puts forward substantive implications
for academics and management.
© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd.
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0).

(AHP) to construct the service quality criteria of thecounseling service system, which can be used by systemmanagement units as a basis for improving system gaps
and improving user satisfaction and willingness to use.

Introduction

The counseling service system developed by the Tainan
Students Guidance and Counseling Center is a casemanagement website, which is intended to track students’development effects and achieve the role of transmitting
information between the center and schools. However, past
research has found that users’ satisfaction and willingness
to use the system are both low [1],[2]. The interface of thecounseling service system includes Instant Message,Individualized Intervention, Resource Integration
Application, Other Services Application, Service Time
Inquiry, and Confirmation [2]. In view of past related
research, there are still opinions from other relevant units
that have not been presented. This study attempts to fill inthe evaluation basis for the service quality of thecounseling service system by adding expert suggestions
from different backgrounds. This study used the modified
Delphi method (MDM) and analytic hierarchy process

2.

Theoretical Background

2.1

System service quality

The service quality point of view regards the organizationas having the goal of providing customers with highquality services and has a collection of multiple processes[3]. This study defines system service quality as "theinformation system provider's delivery of services to users
through the website environment and the users’ overall
evaluation of the service".
In the past, there have been many studies to modify themeasurement indicators of online or website service
quality. For example, the Electronic Retailing Quality
Scale (E-TailQ) uses 14 items to evaluate the quality of etailing commerce, including website design, Fulfilment /Corresponding author’s E-mail: liming@mail.cjcu.edu.tw, harry2356969@gmail.com

369

Reliability, Privacy / Security and customer service, andother four dimensions [4]. There are also the e-Core
Service Quality Scale (ES-Qual) (including Efficiency,Fulfillment, System availability, and Privacy) and the eRecovery Service Quality Scale (E-RecS-Qual) (including
Responsiveness, Compensation, and Contact), both ofwhich are a scale that can measure the performance ofwebsite services [5].
In summary, both the E-TailQ scale and the ES-Qual scale
are derived from rigorous statistical analysis and empiricalresearch and have been verified in subsequent studies[6],[7]. Therefore, this study adopts the structure of the twoas the basis of the prototype of the system service quality
index.
2.2

service quality sub-criteria.
Materials and Methods

3.

This study uses MDM and AHP to conduct a two-stage
expert questionnaire survey. The interviewees are experts
and users who use the counseling service system and arefamiliar with the its functions. The first stage of thequestionnaire is a two-round MDM screening the criteria of
the system service quality, and the second stage uses AHP
to find the criteria and their weights.
3.1

Subjects of the expert questionnaire

The expert scale of the research questionnaire has thelowest group error and the highest credibility when there
are more than 10 people [10]. The experts in this study
belong to a group with high homogeneity, so 16 experts areused as the expert group. The test subjects include
professional guidance counselors, senior high school,
junior high school, and elementary school full-timeteacher-counselor. The seniority ranges from three to morethan ten years. For this research topic, the tested subjects
have rich experience in system use.

Prototype of system service quality index

This study refers to the system service quality [8],[9] andthe related research of elementary school full-time teachercounselor on the counseling service system [1],[2], andinitially summarizes the system service quality index as
fivecriteria:
Tangible,Reliability, Assurance,Responsiveness, and Empathy, and develops 23 system

Table 1. Selection results of the second round of the revised Delphi questionnaire
Criteria

Sub-criteria

A1 The proper layout
A2 Information on all options is available
A3 Easy to learn how to use
A4 The function is simple and convenient tooperate
A5 Easy to find the information you need
B1 Web design & layout are rich and complete
Reliability
B2 Web design & layout meet the needs of thework
B3 Improve the confidentiality mechanism
B4 Stable use and operation
B5 Web-to-web connections are fast and smooth
C1 Helps communicate work matters
Assurance
C2 Helps to improve work efficiency
C3 Improve the efficiency and convenience ofpaperwork
C4 Reduce the cost of physical information orarchives
Responsiveness D1 Be able to quickly resolve questions andspecial needs
D2 Can inform the service response time
D3 Have good professional training
D4 There are regular maintenance personnel
D5 Website system failures are dealt withimmediately
E1 The expression of the web page is clear
Empathy
E2 Protection of information
E3 Be able to understand the problem quickly
E4 Have a good service attitude

TangibleFirst round

Second round

Mean
4.00
4.19
4.56
4.75

Q.D.
0.00
0.50
0.50
0.38

Q.D.
0.38
0.50
0.50
0.00

4.31
3.88
4.44

0.50 0.95 Yes
0.38 0.81 Yes
0.50 0.63 Yes

4.56
4.19
4.75

0.50 0.63 Yes
0.50 0.75 Yes
0.00 0.58 Yes

Use
Use
Use

4.44
4.31
4.00
3.81
4.06
4.31

0.50
0.50
0.88
0.88
0.50
0.50

No
Yes
No
Yes
Yes
No

4.75
4.69
4.31
4.00
4.38
4.56

0.38
0.50
0.50
0.75
0.50
0.50

Yes
Yes
Yes
Yes
Yes
Yes

Use
Use
Use
Use
Use
Use

4.44

0.50 0.63 Yes

4.44

0.50 0.81 Yes

Use

3.88

1.00 1.09 No

4.19

0.50 0.66 Yes

Use

3.50
4.06
4.13
4.06

0.50
0.88
0.50
0.38

0.73
0.77
0.72
0.68

Yes
Yes
Yes
Yes

3.69
4.06
4.38
4.25

0.50
0.38
0.50
0.50

0.95
0.68
0.81
0.58

Yes
Yes
Yes
Yes

Unuse*
Use
Use
Use

4.50
4.63
4.06
4.00

0.50
0.38
0.50
0.75

0.52
0.81
0.85
0.73

Yes
Yes
Yes
Yes

4.63
4.88
4.38
4.13

0.50
0.00
0.50
0.38

0.50
0.34
0.72
0.62

Yes
Yes
Yes
Yes

Use
Use
Use
Use

*The average score < 4370

S.D.
0.63
0.75
0.51
0.45

1.03
0.70
1.03
0.91
0.93
1.01

Consistency Mean
Yes
3.88
Yes
4.56
Yes4.63
Yes
4.81

S.D.
0.81
0.51
0.62
0.40

0.45
0.48
0.87
0.73
0.50
0.51

Results

Consistency
Yes
Unuse*
Yes
Use
Yes
Use
Yes
UseUsing MDM to screen the criteria of system service
quality
The MDM uses a literature review to organize and develop
a prototype questionnaire, which replaces the open
questionnaire of the typical Delphi method as the firstsurvey [11]. Based on the use of the MDM, it can fullyreflect the experts’ opinions, brainstorming, and highaccuracy. [12].
This study used three criteria: mean, quartile (Q.D.) andstandard deviation (S.D.) to verify the expert consensus[10]. In the average score standard, this study uses theaverage score ≥ 4 as the sub-criteria selection criterion. On
the standard of interquartile difference, when the question
interquartile difference ≤ 0.6, it means that the expert
opinions are highly consistent, and the range of 0.6-1.00
means that the expert opinions are moderately consistent.
In the standard deviation standard, when the standard
deviation of the item is less than 1.0, it means that theexpert opinions are consistent. If the expert opinions ofmore than 85% of the items agree, the questionnaire will
be completed [13].
After two rounds of the modified Delphi questionnaire, theaverage score of the two sub-criteria did not reach four
points, so they were deleted. After deletion, they became
five criteria and 21 sub-criteria (Table 1).

structure into three layers. The first layer is the goal of theresearch, the second layer is the main dimensions ofsystem service quality indicators, which are five criteria ofTangible, Reliability, Responsiveness, Empathy andAssurance, and the third layer contains 21 system service
quality criteria. The measurement index design of thisAHP questionnaire adopts the nine grades suggested by
Saaty for pairwise comparison, and Expert Choice 11.5 isused for data calculation and analysis to obtain the element
weight value of the questionnaire selection.

3.2

3.3

4.

Data Analysis and Results

4.1

Weight analysis of primary and secondary criteria

After calculating the weights of the five criteria and 21
sub-criteria selected by the system service quality index,
the C.I. and C.R. of all items are less than 0.1, which meetsthe consistency requirements. The criteria have the highest
relative weight of Reliability, showing that the key to theservice quality index of the counseling service system is tomeasure Reliability. Under the five criteria, each is theservice quality index with the highest weight in the subcriteria. “The functions are simple and convenient tooperate”, “Improve the confidentiality mechanism”,“Helps to improve work efficiency”, “Be able to quickly
resolve questions and special needs”, and “Protection ofinformation” (Table 2).

Establishing AHP hierarchy and conduct expert
questionnaire

Table 2. Weights of criteria and Sub-criteria

AHP is mainly used in uncertain situations and decisionmaking problems with multiple evaluation criteria andprovides a systematic decision-making mode to reduce therisk of decision-making errors [14]. AHP can provide
scientific and quantitative comparison results and use theprofessional evaluation of experts to generate variablefactors that can be ranked. Therefore, this study uses AHP
to evaluate the weight of system service quality criteria.
Regarding the use of AHP, the criteria selected in this study
have avoided unreasonable or incorrect correlations
between criteria, so as to improve the accuracy of the AHPresults.
The consistency index (C.I.) and the consistency ratio(C.R.) [14] are used to verify the consistency of the paired
comparison matrix. When C.I. = 0, it means that the before
and after judgments are completely consistent, and C.I.
<0.1 is an allowable error. When C.R. <0.1, theconsistency of the matrix is satisfactory.
After two rounds of a modified Delphi questionnaire
survey, this study divided the system service quality index

Criteria

Weight Sub(Rank) criteria

Weight(Rank)

Integrated
weight (Rank)

Tangible

0.257
(2)

Reliability

0.310(1)Assurance

0.214(3)0.132 (18)0.326 (4)0.362 (2)0.179 (14)0.137 (17)0.290 (9)0.309 (6)0.154 (16)0.111 (19)0.207 (12)0.487 (1)0.195 (13)0.111 (20)0.314 (5)0.294 (8)0.158 (15)0.234 (11)0.303 (7)0.359 (3)0.235 (10)0.033 (13)0.082 (4)0.091 (3)0.045 (8)0.049 (7)0.104 (2)0.110 (1)0.055 (6)0.040 (10)0.032 (15)0.076 (5)0.031 (16)0.017 (20)0.040 (9)0.037 (12)0.020 (19)0.030 (17)0.032 (14)0.038 (11)0.025 (18)

Responsiveness 0.112(4)Empathy371

0.108(5)A2
A3A4A5B1
B2
B3B4B5C1C2C3C4D1
D3
D4
D5E1
E2
E3

E4

4.2

0.103 (21)0.011 (21)and maintain case management websites. With a view tothe system in the development process, the users’ point ofview and the importance of subsequent maintenance andmanagement can be considered.

The overall weight analysis of the criteria

The primary and secondary criteria were comprehensivelyevaluated, and the overall weight was the highest for the
top five sub-criteria, in order to “Improve theconfidentiality mechanism”, “Web design & layout meetthe needs of the work”, “The function is simple andconvenient to operate”, “Easy to learn how to use”, and“Helps to improve work efficiency” (Table 2). Since
reliability and tangibility have the highest weight in thecriteria, the sub-criteria under the criteria also have thehigher overall weight.
In summary, from the results of the weight analysis, wecan see the importance of the criteria. Experts attach great
importance to the data security and rigor of the online
system, followed by whether the system actually helps thework. Past research has also shown that Design,
Availability, Security and Privacy are positively related tooverall electronic service quality [6],[7],[9]. This study isdifferent from the past research purposes of shopping orsocial networking sites, but non-profit sites as the object ofdiscussion, using the conclusion is roughly the same.
Overall, system security and compliance with workrequirements are the keys to affecting system service
quality. Since the counseling service system used byTainan City at this stage has not been universally
recognized by users, the results of this study can be usedby its management and maintenance units as a direction
for improvement.
5.

Conflicts of Interest
The author declares no conflicts of interest.
References
[1]

[2]

[3]

[4]

[5]

[6]

[7]

Conclusions and Suggestions[8]

In addition to the counseling service system being used in
Tainan City, counselors in other counties, cities, orinstitutions also use similarly functional student
counseling record systems. This study used MDM andAHP to collect the opinions of experts from various units
to obtain indicators of system service quality. Practicalsuggestions for improving system service quality include
providing users with complete authority descriptions,
reducing unnecessary items to fill in, and improving
Reliability; the layout is mainly concise, avoiding
complicated operation steps to improve Tangible; throughlinking related systems, common data is automaticallybrought in to improve work efficiency and help improve
Assurance. The system service quality indicators of thisstudy are not only provided to the system management unitof Tainan City as reference indicators for regular
maintenance, but can be used as a basis for othergovernment agencies or school institutions to construct

[9]

[10]

[11]

[12]

[13]

[14]

372

Liu, H.-H. (2021). A Study of the Counseling Service
System Satisfaction for Teacher-Counselor in Tainan
Elementary School. Journal of Robotics, Networking andArtificial Life, 8(3), 197-200.
Liu, H.-H. (2022). A Study of Factors Affecting
Acceptance of Counseling Service System: Service Users’
Perspective. Journal of Robotics, Networking andArtificial Life, 9(1), 54-58.
Zeithaml, V.A., Parasuraman, A., & Malhotra, A. (2000).
A Conceptual Framework for Understanding E-Service
Quality: Implications for Future Research and ManagerialPractice. Cambridge, Mass: Marketing Science Institute.
Wolfinbarger, M., & Gilly, M.C. (2003). eTailQ:
Dimensionalizing, measuring and predicting etail quality.
Journal of Retailing, 79(3), 183-198.
Parasuraman, A., Zeithaml, V.A., & Malhotra, A. (2005).
E-S-QUAL: A multiple-item scale for assessing electronicservice. Journal of Service Research, 7(3), 213-233.
Ghosh, M. (2018). Measuring electronic service quality inIndia using E-S-QUAL. International Journal of Quality &Reliability Management, 35(2), 430-445.
Lopes, E.L., Freire, O., & Lopes, E.H. (2019). Competing
scales for measuring perceived quality in the electronic
retail industry: A comparison between E-S-Qual and ETailQ. Electronic Commerce Research and Applications,34, 100824.
Laxmi M. (2020). A Study of Reviewing Country andIndustry Specific E-Service Quality Dimensions. Journalof Critical Reviews, 7(16), 3741-3748.
Rita P., Oliveira T., & Farisa, A. (2019). The impact of eservice quality and customer satisfaction on customer
behavior in online shopping. Heliyon, 5(10), e02690.
Brooks, K.W. (1979). Delphi Technique: ExpandingApplications. North Central Association Quarterly, 53(3),377-385.
Murry, J.W., & Hammons, J.O. (1995). Delphi: A Versatile
Methodology for Conducting Qualitative Research. The
Review of Higher Education, 18(4), 423-436.
Thangaratinam, S., & Redman, C. (2005). The DelphiTechnique. The Obstetrician & Gynaecologist, 7(2), 120125.
Holden, M.C., & Wedman, J.F. (1993). Future Issues ofComputer-mediated Communication: The Results of aDelphi Study. Educational Technology Research andDevelopment, 41(4), 5-24.
Saaty, T.L. (1980). The Analytic Hierarchy Process. New
York: McGraw-Hill.

Authors Introduction
Dr. Li-Min Chuang
He received the Ph.D. degree from theNational Cheng Kung University. He
is currently an associate professor anddepartment head at the Department ofInternational Business at Chang Jung
Christian University, Taiwan. His research interests focus on strategic management, innovation management, andservice innovation.
Mr. Hsiu-Hao Liu
He received his Master’s degree fromthe Department of Psychology,
National Chung Cheng University,
Taiwan in 2015. He is currently aDoctoral course student in Chang Jung
Christian University, Taiwan. His
researchinterests
includetheapplication of information technology
to education and counseling.

373