# Toward a Comprehensive Understanding and Evaluation of the Sustainability of E-Health Solutions
## 6 E-Health Sustainability Assessment Method
### Sustainability Evaluation Tools  
The elements of each aspect are assessed by either checklist or survey questions. Both checklists and surveys have a five-point Likert scale. The scales ranges from strongly agree, agree, neutral, disagree, to strongly disagree, which is equivalent to the numerical scale 5-1, respectively. The questions that were used to assess the model elements were gathered from different resources found in the literature. It is worth mentioning that most of the resources focus on evaluating software products in general, and in this work, they have been adapted for the context of e-health solutions. To illustrate, some of the questions in the evaluation tools were altered in terms of the wordings to fit the context of e-health solution evaluation. Furthermore, the questions' focus of some aspects was deduced from the literature, and based on that, the questions were developed by the author. Table 13 shows how each aspect's questions were customized. In addition, some aspects require regular evaluation (e.g., every year or after a major change/update on the e-health). The importance of regular evaluation is that some changes might happen to the e-health solution after the assessment phase, and thus the predicted sustainability level may change as a result. Furthermore, with the rapid changes in demands \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Dimensions** & **Aspects** & \begin{tabular}{c} **Type of** \\ **Evaluation Tool** \\ \end{tabular} & **Customization** & \begin{tabular}{c} **Evaluation** \\ **Frequency** \\ \end{tabular} & **References** \\ \hline \multirow{4}{*}{Technology} & System quality & Checklist & Change of wording & \begin{tabular}{c} On system change \\ or update \\ \end{tabular} & [51; 52; 53] \\ \cline{2-6}  & Data quality & Checklist & Change of wording & \
\begin{tabular}{c} On system change \\ or update \\ \end{tabular} & [54] \\ \cline{2-6}  & Hardware quality & Checklist & Developed \({}^{*}\) &
\begin{tabular}{c} On system change \\ or update \\ \end{tabular} & [34] \\ \hline \multirow{4}{*}{Organization} & Support and \\  & commitment & Checklist & Change of wording & Annually & [55] \\ \cline{2-6}  & Organizational \\ structures and strategy & Checklist & Change of wording & Once & [19; 55] \\ \cline{2-6}  & Planning and policies & Checklist & Developed \({}^{*}\) & Annually & [55; 56; 57; 58] \\ \cline{2-6}  & Stakeholders & Checklist & Developed \({}^{*}\) & Annually & [19; 55; 56; 59] \\ \cline{2-6}  & Culture of electronic \\ information use & Checklist & Change of wording & Once & [60] \\ \cline{2-6}  & Outcome identification & Checklist & Developed \({}^{*}\) & Annually & [28] \\ \cline{2-6}  & Demand and need & Checklist & Developed \({}^{*}\) & Annually & [55] \\ \hline \multirow{4}{*}{Resources} & Human resources & Checklist & Developed \({}^{*}\) & Annually & [19; 56] \\ \cline{2-6}  & Technical support and \\  & training & Checklist & Change of wording & Annually & [19; 55; 61; 62] \\ \cline{2-6}  & ICT infrastructure & Checklist & Change of wording & Annually & [35; 55; 59; 62] \\ \cline{2-6}  & Financial resources & Checklist & Developed \({}^{*}\) & Annually & NA \\ \hline \multirow{4}{*}{Social} & Continuance usage \\  & intention & Survey & Change of wording & Periodically & [29] \\ \cline{2-6}  & User behavior & Survey & Change of wording & Periodically & [63; 64; 65] \\ \cline{2-6}  & Sustainable funding & Checklist & Developed \({}^{*}\) & Annually & [7; 26] \\ \cline{2-6}  & Affordability of \\ \cline{2-6}  & e-solution & Checklist & Developed \({}^{*}\) & Once & [35] \\ \cline{2-6}  & Financial benefits & Checklist & Developed \({}^{*}\) & Annually & [47] \\ \hline \hline \end{tabular}  
* Developed by the author based on the aspect’s definition.  
\end{table} 
Table 13: Types of evaluation tools of each sustainability aspect. on healthcare environments, the goals and needs might change quickly, hence, a deployed e-solution might not be the optimal solution anymore.