## 5 Validation
The methodology was applied to HCAHP data obtained from CAHPS Database and the generated recommendations were validated using statistical tests. In the presented case study in this work, the proposed methodology, extracted nineteen associations from the HCAHPS dataset of a hospital with 2652 records. Ten associations out of nineteen were validated through statistical methods of chi-squared independence test, and odds ratio z-test, which shows the reliability of the proposed recommendation system. The proposed recommendation system provides knowledge that may be hidden to an expert analyzing the surveys and rectifies the need for a subject-matter expert. The analysis approach is designed specifically for the format of the standard HCAHPS survey; however, it can be extended to other domains in which customer survey plays an important role. The future work of this study will focus on three aspects:

1. **More extensive data collection:** Our long-term goal is to assess how the recommendations produced by our system can improve patients' loyalty and result in saving costs and time in the long run. Using the preliminary results outlined in this paper, our goal is to obtain a more comprehensive data set which includes data on whether the patients have come back to the hospital if medical services were needed, and to examine the relationship between customer loyalty and their satisfaction identifiers. 2. **Handling skip questions:** In this study, a single imputation method based on K-nearest neighbor (KNN) is used to impute missing values. Other popular approaches, such as multiple imputations by chained equations (MICE) [19], should be explored in future for imputing both categorical and continues variables. Also, other approaches for handling skip questions should be examined to better distinguish between non-applicable and missing data. 3. **Alternative distance measures for K-means:** In this study, we used Euclidean distance for clustering. Euclidean distance is typically used for continuous data where data are seen as points in the Euclidean space. Since HCAHPS data consist of mixed numeric and categorical variables, we should examine other types of distance measures such as cosine, Jaccard, Overlap, Occurrence Frequency, etc. [20] and compare the quality of recommendations produced by each measure. In addition, since there are two layers of clustering and the intrinsic characteristics of data points in each layer vary, and a different distance function can be used for each layer. The CAHPS(r) data used in this analysis were provided by the CAHPS Database. The CAHPS Database is funded by the U.S. Agency for HealthCare Research and Quality (AHRQ) and administered by Westat under Contract No. HHSA290201300003C. [PERSON] studied conception and design; [PERSON] and [PERSON] and [PERSON] acquired data; All authors analysed and interpreted data and evaluated results; [PERSON] and [PERSON] and [PERSON] drafted the manuscript; [PERSON] and [PERSON] and [PERSON] did critical revisions. The authors declare no conflict of interest. ## Appendix A

The HCAHPS survey has been changed through the time. In this appendix A, we brought the questionnaire of March 2012. _Appl. Sci._**2017**, 7, 966

YOUR CARE FROM DOCTORS

**5. ** During this hospital stay, how often did doctors treat you with courtesy and respect? **

\({}^{1}\) **Never**

\({}^{2}\) **Sometimes**

\({}^{3}\) **Usually**

\({}^{4}\) **Always**

**6. ** During this hospital stay, how often did doctors listen carefully to you? **

\({}^{1}\) **Never**

\({}^{2}\) **Sometimes**

\({}^{3}\) **Usually**

\({}^{4}\) **Always**

**7. ** During this hospital stay, how often did doctors explain things in a way you could understand? **

\({}^{1}\) **Never**

\({}^{2}\) **Sometimes**

\({}^{3}\) **Usually**

\({}^{4}\) **Always**

**8. ** During this hospital stay, how often were your room and bathroom kept clean? **

\({}^{1}\) **Never**

\({}^{2}\) **Sometimes**

\({}^{3}\) **Usually**

\({}^{4}\) **Always**

**9.