<figure><img src="image_1.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Sustaining Continuous Improvement of a Higher Health Education Service through Analytical Methodologies for Determining Customer Satisfaction

Alfonso Maria Ponsiglione<fnref n="1" /> and Angelo Rosa<fnref n="2" />, Teresa Angela Trunfio<fnref n="3" />*, Eliana Raiola<fnref n="4" />, Giuseppe Longo<fnref n="4" />, Maria Triassi<fnref n="5" />,<fnref n="6" />, Francesco Amato<fnref n="1" />

teresa.trunfio@gmail.com

## Abstract

The transformation of hospitals into companies has made the concept of quality even more important. The hospital to compete must be able not only to provide good service, but also to put in place a whole series of actions that lead the user throughout the service delivery process. To do this, it requires skills and training programs that can verify and evaluate the services offered to the citizen/patient. In this study, the aim is to evaluate the level of quality achieved in the delivery of a High Health Education service provided at the Biotechnology Center of the A.O.R.N. "A. Cardarelli" in Naples. The main method adopted for collecting the desired information was a questionnaire, identified as the tool for excellence for quantitative measurement. The questionnaires were submitted to 341 learners who attended the following Higher Education courses. Then, analysis was carried out using two different methodologies: the constant sum scale and pairwise comparison. The decomposition of the problem into criteria and sub-criteria of interest made it possible to identify the degree of importance of each requirement. In general, learners preferred attributes such as the duration, preparation and interpersonal skills of the teacher and fewer elements of logistics. A training program, of whatever nature, cannot ignore an evaluation aimed at determining its effectiveness and efficiency. In this study, we intend to propose analysis strategies based on simple collection tools, such as questionnaires, that help to understand learners' needs. In this way, changes in these needs over time can be monitored in order to activate a process of continuous service improvement.

**Keywords:** Healthcare, Quality, High Health Education service, Constant sum scale, Pairwise comparison

## 1. Introduction

Healthcare is a very complex area and quality improvement can be a common paradigm to address needs [1]. However, the definition and measurement of quality in this field is extremely complicated. Different healthcare characteristics such as intangibility, heterogeneity and simultaneity combined with the intricate nature of healthcare and its many participants with own interests are just a few examples that define its complexity [2]. In addition, quality standards are more difficult to establish when delivering a service. The service is delivered differently by different operators, elements such as experience, individual skills and personality are affected [3]. Moreover, they are produced and consumed at the same time and the customer can only judge the quality at the end [4].

According to Mosadeghrad [2], the definition of quality is also linked to the standardisation of processes. Various management strategies can be deployed to search for inefficiencies and optimise processes. One example is Lean Six Sigma, which from industry has been applied in various areas of healthcare, such as the management of hospital infections [5, 6] or the optimisation of care pathways

<footnote n="1">¹Department of Electrical Engineering and Information Technology, University of Study of Naples “Federico II”,
Naples, Italy</footnote>

<footnote n="2">²Department of Management, Finance and Technology, University LUM “Giuseppe Degennaro” of Casamassima,
Puglia, Italy</footnote>

<footnote n="3">³Department of Advanced Biomedical Sciences, University of Naples “Federico II”, Naples, Italy</footnote>

<footnote n="4">⁴A.O.R.N. “A. Cardarelli”, Naples, Italy</footnote>

<footnote n="5">⁵Department of Public Health, University of Naples “Federico II”, Naples, Italy</footnote>

<footnote n="6">⁶Interdepartmental Center for Research in Healthcare Management and Innovation in Healthcare (CIRMIS), University of Naples “Federico II”, Naples, Italy</footnote>
[7, 8]. To support these strategies, data-driven techniques, e.g. regression [9, 10], classification [11, 12] and analysis models, which have already been validated in other applications, such as the study of biomedical data [13, 14], support for diagnosis and therapy processes [15, 16] or hospital resource management [17, 18], are indispensable. These same techniques can be successfully applied to the quality assessment process.

Another key element is the training of healthcare personnel. The education system in health care needs continuous development and improvement because of the important role played by practitioners. Any deficiency or omission in education will result in operators who are unable to adapt to the rapid and competitive changes typical of this sector [19]. To ensure this development, it is crucial to know the opinion and needs of customers. The voice and opinion of customers influence quality improvement and provide an opportunity for organisational learning [20]. Furthermore, it is the provision of quality services that creates loyal consumers. Borishade et al. [21] demonstrated a significant association between service quality and student loyalty. However, this relationship is mediated by their satisfaction. Collecting information on learner satisfaction is therefore indispensable both for the optimisation of the service and to ensure continuity.

In this study, the opinions of 241 learners of a Higher Health Education service provided at the Biotechnology Center of the A.O.R.N. 'A. Cardarelli' in Naples (Italy) were collected. In particular, the learners were given questionnaires written in different ways according to the methodology under investigation. In this paper, 166 questionnaires will be analysed with two different methodologies; the Constant Sum Scale and the Classic Pairwise Comparison. The aim is to detect learners' opinions and determine the most suitable methodology to gather information and support improvements.

## 2. Methodology

In this study, data on satisfaction with a Higher Health Education service provided at the Biotechnology Center of the A.O.R.N. 'A. Cardarelli' in Naples were processed. Using questionnaires, the opinions of 166 learners who attended the Higher Education courses were collected. In this paper, in accordance with what was proposed by Y. M. Wang and K. S. Ching [22], different questionnaire formats were used without the use of conversions, and the processing of the data, as obtained from each questionnaire, was conducted according to two methodologies: the constant sum scale and pairwise comparison. The sub-dimensions, or sub-criteria, represent the Customer Requirements (CRs). In particular, structure, teacher and organisation were chosen as the main dimensions (criteria). The following sub-dimensions (sub-criteria) were associated with the structure dimension:

- Location: meaning the location of the building both within the hospital, and in the vicinity of car parks, subways and other means of transport that can enable it to be reached;

- Equipment: meaning the presence and suitability of the equipment provided for the purposes of achieving the objectives of the training courses;

- Comfort of the environments: meaning those aesthetic characteristics that make the available spaces pleasant and comfortable.

The following sub-dimensions were associated with the teacher dimension:

- Preparation: meaning the adequacy of the teacher's level of training and preparation for teaching purposes;

- Interpersonal skills: meaning the teacher's ability to relate to learners by interpreting their requests and providing clear and comprehensive answers;

- Effectiveness: meaning the teacher's ability to deal with topics aimed at achieving the course objectives.
Finally, organisation is the criterion that includes the general aspects that characterise the entire training process that can be better defined through the following sub-criteria:

- Cost: this means the cost incurred for the course under consideration;

- Duration: means the duration of the course under examination;

- Content: means the correspondence of the topics covered in the course under examination to the user's expectations.

The user preferences are only an estimate of the importance weights $W = (w1, ..., wn)^T$ relative to the CRs. Below is the detail of the adopted methodologies.

## 1.1. Constant Sum Scale

In this type of questionnaire the learner is asked to allocate 100 points, in proportion to the degree of importance, from 0 not at all important to 100 only important, within each of the identified levels. The weights can be linked by three types of relationships:

- $w_j^{(1)} > w_k^{(1)}$ close order of preference;

- $w_j^{(1)} \ge w_k^{(1)}$ weak preferential order;

- $w_j^{(1)} = w_k^{(1)}$ indifferent preferential order.

## 1.2. Pairwise Comparison

This technique involves comparing a set of alternatives $W = (w1, ..., wn)$, two at a time. For each comparison, learners were asked to determine which of the two compared parameters had, in their experience, greater importance in determining service quality. Specifically, learners express their judgement within the semantic scale shown in the Table.

<table><caption>Table 1. Saaty's Fundamental Scale</caption><thead><tr><td>Intensity of importance</td><td>Definition</td><td>Explanation</td></tr></thead><tbody><tr><td>1</td><td>Equal Importance</td><td>The two elements contribute equally to the goal.</td></tr><tr><td>3</td><td>Moderate Importance</td><td>The judgment is slightly in favour of one element over the other.</td></tr><tr><td>5</td><td>Strong Importance</td><td>The judgment is decidedly in favour of one element over the other.</td></tr><tr><td>7</td><td>Very Strong Importance</td><td>The predominance of the element is amply demonstrated.</td></tr><tr><td>9</td><td>Extreme Importance</td><td>The evidence in favour of an element is of the highest order.</td></tr><tr><td>2, 4, 6, 8</td><td>Intermediate value between adjacent judgments</td><td>When a compromise is needed.</td></tr><tr><td>Reciprocal</td><td colspan="2">If activity i takes on a certain value when it is compared with activity j, when j is compared with i it will take on the corresponding reciprocal value.</td></tr></tbody></table>

The results of these comparisons are placed within a pairwise comparison matrix $A = (a_{ij})$, such that $A \subset W \times W$. The element $a_{ij}$, called the dominance coefficient, represents the preference ratio of alternative $w_i$ over alternative $w_j$. The dominance coefficients of the matrix $A = (a_{ij})$, must respect the following properties:

- $\forall a_{ij} > 0 \forall i, j \in \{1, ..., n\}$

- $a_{ii} = 1 \forall i \in \{1, ..., n\}$

- $a_{ij} * a_{ji} = 1 \forall i, j \in \{1, ..., n\}$

- $a_{ij} * a_{jk} = a_{ik} \forall i, j, k \in \{1, ..., n\}$

The pairwise comparison matrix provided by the learners in the group is shown below:
$$A^{(2)} = \begin{bmatrix} a_{11}^{(2)} & a_{12}^{(2)} & \cdots & a_{1n}^{(2)} \\ a_{21}^{(2)} & a_{22}^{(2)} & \cdots & a_{2n}^{(2)} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1}^{(2)} & a_{n2}^{(2)} & \cdots & a_{nn}^{(2)} \end{bmatrix}$$

This matrix must fulfil the following conditions:

* $a_{ij}^{(2)} = 1 \forall i \in \{1, \dots, n\}$

* $a_{ij}^{(2)} = 1 / a_{ji}^{(2)} > 0 \forall i,j \in \{1, \dots, n\}$

* $a_{ij}^{(2)} a_{jk}^{(2)} = a_{ik}^{(2)} \forall i,j,k \in \{1, \dots, n\}$

The logarithmic geometric mean will be used to construct the matrix.

### 3. Results

A portion of the questionnaires was eliminated from the final analysis as it was assumed to be subject to the response effect phenomenon. The 166 questionnaires were analysed as follows:

- 100 with the Constant Sum Scale method;

- 66 with the Classic Pairwise Comparison.

As far as the Constant Sum Scale (CSS) is concerned, Figure 1 shows the results obtained.

<figure><img src="image_2.png" /><figcaption>Figure 1. Summary of data collected with CSS.</figcaption></figure>

The average scores obtained for Methodology 1, $w_1^{(1)} = 30$, $w_2^{(1)} = 40$ e $w_3^{(1)} = 30$, show how learners give more weight to the Organisation and less to teachers and structure. Figures 2, 3 and 4 show the results obtained for each dimension.
<figure><img src="image_3.png" /><figcaption>Figure 2. Data collected for the CRs of the Teacher dimension with CSS.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 3. Data collected for the CRs of the Organisation dimension with CSS.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>Figure 4. Data collected for the CRs of the Structure dimension with CSS.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 4. Data collected for the CRs of the Teacher dimension with CSS.</figcaption></figure>

With regard to the average values for each area, learners attributed a higher weight to the teacher's interpersonal skills, the course content and the classroom equipment. The least weight was attributed to the cost of the course followed by the location of the classroom. As far as the Classic Pairwise Comparison (CPC) is concerned, the overall result is shown in Figure 5.
<figure><img src="image_7.png" /><figcaption>Figure 5. Summary of data collected with CPC.</figcaption></figure>

Using the logarithmic geometric mean for the calculation of the elements aij, in the case of the Teacher/Structure comparison we obtained a13 = 3.25 (a31 = 0.31), for Organisation/Structure we obtained a23 = 2.39 (a32 = 0.42) and finally between Organisation/Teacher a21 = 1.08 (a12=0.93). From a general point of view, therefore, learners gave more weight to the teacher and the organisation rather than the structure while between organisation and teacher the former is given more weight. Figures 6, 7 and 8 show the intermediate results for each CRs.

<figure><img src="image_8.png" /><figcaption>Figure 6. Data collected for the CRs of the Teacher dimension with CPC.</figcaption></figure>
For the teacher dimension, we obtained in the case of the comparison Interpersonal Skills/Efficacy $a_{13} = 1.47$ ($a_{31} = 0.68$), in the case of the comparison Preparation/Efficacy $a_{23} = 2.72$ ($a_{32} = 0.37$) and finally in the case of the comparison Preparation/Interpersonal Skills $a_{21} = 2.16$ ($a_{12} = 0.46$).

<figure><img src="image_9.png" /><figcaption>Figure 7. Data collected for the CRs of the Organisation dimension with CPC.</figcaption></figure>

For the organisation dimension, on the other hand, the following were obtained in the case of the Duration/Contents comparison $a_{12} = 1.68$ ($a_{21} = 0.59$), in the case of the Cost/Content comparison $a_{32} = 0.84$ ($a_{23} = 1.19$) and in the case of the Cost/Duration comparison $a_{31} = 0.81$ ($a_{13} = 1.23$).

<figure><img src="image_10.png" /><figcaption>Figure 8. Data collected for the CRs of the Structure dimension with CPC.</figcaption></figure>

Finally, the following partial results were obtained for the Structure dimension: in the case of the Location/Equipment comparison $a_{31} = 0.72$ ($a_{13} = 1.39$), in the Comfort/Equipment comparison
$a23 = 1.70$ ($a32 = 0.59$) and in the case of the Comfort/Equipment comparison $a21 = 1.15$ ($a12 = 0.87$). From these results it can be deduced that learners prefer the comfort of the classroom, the duration of the course and the preparation of the lecturer.

## 4. Discussion and Conclusion

In this study, two different methods, namely the constant sum scale and pairwise comparison, were used to collect the opinions of the learners of a High Health Education service provided at the Biotechnology Center of the A.O.R.N. "A. Cardarelli" in Naples (Italy). The results show how the pairwise comparison better delineates the main elements of interest for the learners from a general point of view. Specifically, in the three areas identified, teacher, organisation and structure, while with the constant sum scale we only succeed in defining how the organisation is the dimension with the greatest weight, with the pairwise comparison we also manage to conclude that the teacher assumes a significant relevance with respect to the structure ($a_{13} = 3.25$). The intermediate results, however, showed a different picture in the two cases. In the first case, the learners attached greater weight to the teacher's interpersonal skills, the course content and the classroom equipment, while in the second to classroom comfort, the duration of the course and the teacher's preparation. These differences could be related to the different data collection methodology, especially in the second case where learners had to make a choice between the two sub-criteria.

This work is not without its limitations. In particular, only two methodologies are analysed considering a small number of learners. In addition, critical issues are not identified and improvement strategies are not adopted to monitor satisfaction levels over time. Future developments, in addition to addressing the overcoming of these limitations, will also involve the use of synthesis techniques, such as the Linear Goal Programming technique, to recompose the hierarchy realised based on the Analytic Hierarchy Process [23] in order to derive the final synthesis result of the survey.
References

[1] Aggarwal, A., Aeran, H., & Rathee, M. (2019). Quality management in healthcare: The pivotal desideratum. Journal of Oral Biology and Craniofacial Research, 9(2), 180-182.

[2] Mosadeghrad, A. M. (2013). Healthcare service quality: towards a broad definition. International journal of health care quality assurance.

[3] Mosadeghrad, A. M. (2012). Towards a theory of quality management: an integration of strategic management, quality management and project management. International Journal of Modelling in Operations Management, 2(1), 89-118.

[4] Mosadeghrad A. M. (2014). Factors influencing healthcare service quality. International journal of health policy and management, 3(2), 77-89. https://doi.org/10.15171/ijhpm.2014.65

[5] Ferraro, A., Centobelli, P., Cerchione, R., Cicco, M. V. D., Montella, E., Raiola, E., ... & Improta, G. (2020). Implementation of lean practices to reduce healthcare associated infections. International Journal of Healthcare Technology and Management, 18(1-2), 51-72.

[6] Cesarelli, G., Montella, E., Scala, A., Raiola, E., Triassi, M., & Improta, G. (2020). DMAIC Approach for the Reduction of Healthcare-Associated Infections in the Neonatal Intensive Care Unit of the University Hospital of Naples "Federico II". In European Medical and Biological Engineering Conference (pp. 414-423). Springer, Cham.

[7] Trunfio, T. A., Scala, A., Borrelli, A., Sparano, M., Triassi, M., & Improta, G. (2021, May). Application of the Lean Six Sigma approach to the study of the LOS of patients who undergo laparoscopic cholecystectomy at the San Giovanni di Dio and Ruggi d'Aragona University Hospital. In 2021 5th International Conference on Medical and Health Informatics (pp. 50-54).

[8] Improta, G., Borrelli, A., & Triassi, M. (2022). Machine Learning and Lean Six Sigma to Assess How COVID-19 Has Changed the Patient Management of the Complex Operative Unit of Neurology and Stroke Unit: A Single Center Study. International Journal of Environmental Research and Public Health, 19(9), 5215.

[9] Trunfio, T. A., Scala, A., Giglio, C., Rossi, G., Borrelli, A., Romano, M., & Improta, G. (2022). Multiple regression model to analyze the total LOS for patients undergoing laparoscopic appendectomy. BMC Medical Informatics and Decision Making, 22(1), 1-8.

[10] Scala, A., Trunfio, T. A., De Coppi, L., Rossi, G., Borrelli, A., Triassi, M., & Improta, G. (2022). Regression models to study the total LOS related to valvuloplasty. International journal of environmental research and public health, 19(5), 3117.

[11] Montella, E., Ferraro, A., Sperlì, G., Triassi, M., Santini, S., & Improta, G. (2022). Predictive Analysis of Healthcare-Associated Blood Stream Infections in the Neonatal Intensive Care Unit Using Artificial Intelligence: A Single Center Study. International Journal of Environmental Research and Public Health, 19(5), 2498.

[12] Trunfio, T. A., Borrelli, A., & Improta, G. (2022). Is It Possible to Predict the Length of Stay of Patients Undergoing Hip-Replacement Surgery?. International Journal of Environmental Research and Public Health, 19(10), 6219.

[13] Ponsiglione, A. M., Cosentino, C., Cesarelli, G., Amato, F., & Romano, M. (2021). A comprehensive review of techniques for processing and analyzing fetal heart rate signals. Sensors, 21(18), 6136.

[14] Ponsiglione, A. M., Amato, F., & Romano, M. (2021). Multiparametric investigation of dynamics in fetal heart rate signals. Bioengineering, 9(1), 8.

[15] Improta, G., Mazzella, V., Vecchione, D., Santini, S., & Triassi, M. (2020). Fuzzy logic-based clinical decision support system for the evaluation of renal function in post-Transplant Patients. Journal of evaluation in clinical practice, 26(4), 1224-1234.

[16] Santini, S., Pescapè, A., Valente, A. S., Abate, V., Improta, G., Triassi, M., ... & Filosa, A. (2017, July). Using fuzzy logic for improving clinical daily-care of β-thalassemia patients. In 2017 IEEE international conference on fuzzy systems (FUZZ-IEEE) (pp. 1-6). IEEE.

[17] Improta, G., Converso, G., Murino, T., Gallo, M., Perrone, A., & Romano, M. (2019). Analytic hierarchy process (AHP) in dynamic configuration as a tool for health technology assessment (HTA): the case of biosensing optoelectronics in oncology. International Journal of Information Technology & Decision Making, 18(05), 1533-1550.

[18] Improta, G., Triassi, M., Guizzi, G., Santillo, L. C., Revetria, R., Catania, A., & Cassettari, L. (2012). An innovative contribution to health technology assessment. In Modern advances in intelligent systems and tools (pp. 127-131). Springer, Berlin, Heidelberg.

[19] Saif, N. I. (2014). The effect of service quality on student satisfaction: a field study for health services administration students. International Journal of Humanities and Social Science, 4(8), 172-181.
[20] Carlucci, D., Renna, P., & Schiuma, G. (2013). Evaluating service quality dimensions as antecedents to outpatient satisfaction using back propagation neural network. Health care management science, 16(1), 37-44.

[21] Borishade, T. T., Ogunnaike, O. O., Salau, O., Motilewa, B. D., & Dirisu, J. I. (2021). Assessing the relationship among service quality, student satisfaction and loyalty: the NIGERIAN higher education experience. Heliyon, 7(7), e07590.

[22] Wang, Y. M., & Chin, K. S. (2011). A linear goal programming approach to determining the relative importance weights of customer requirements in quality function deployment. Information Sciences, 181(24), 5523-5533.

[23] Ponsiglione, A. M., Amato, F., Cozzolino, S., Russo, G., Romano, M., & Improta, G. (2022). A Hybrid Analytic Hierarchy Process and Likert Scale Approach for the Quality Assessment of Medical Education Programs. Mathematics, 10(9), 1426.