# Toward a Comprehensive Understanding and Evaluation of the Sustainability of E-Health Solutions
## 4 Results
### The Sustainability of E-Health Solutions
#### 4.2.3 Evaluation/Assessment Methods
However, it worth noting that the mentioned KPIs of all dimensions, other than the technology dimension, were focused on assessing the positive impact of IT systems on the efficiency of the healthcare system. For example, in the environment dimension, the indicators evaluated the reduction in paper use and energy consumption as a result of using the IT system. However, the technological KPIs \begin{table}
\begin{tabular}{p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt}} \hline \hline
**Study** & **Method of Assessment** & **Type of Assessment Methods** & **Source** & **E-Health Solution in Real Context?** & **Evaluated/Validated in Real Context?** \\ \hline \multirow{10}{*}{PS14} & \multirow{10}{*}{Model} & \multirow{10}{*}{Standards} & 1. & Clinical and Laboratory Standards Institute (CLSI) & & \\  & & & 2. & “Institute of Electrical and Electronics Engineers Standards Association (IEEE)” & & \\  & & & 3. & “International Electrotechnical Commission Technical Report (IEC/TR)” & & \\  & & & 4. & “International Organization for Standardization - Technical Report (ISO/TR)” & & \\  & & & 5. & “German Institute for Standardization (DIN) & & \\ \hline \multirow{10}{*}{PS15} & \multirow{10}{*}{Catalog} & \multirow{10}{*}{Cheadist of software quality requirements} & \multirow{10}{*}{Elicited from several sources in software engineering} & \multirow{10}{*}{Connected health apps (e.g., telemedicine, m-health, etc.)} & \multirow{10}{*}{Yes, evaluated} \\  & & & & & & \\ \cline{1-1}  & & & & & \\ \cline{1-1}  & & & & & \\ \cline{1-1}  & & & & & \\ \hline \multirow{10}{*}{PS16} & \multirow{10}{*}{Model} & \multirow{10}{*}{Qualitative feedback and quantifiable scale} & \multirow{10}{*}{Combination of two existing models (CeffRes roadmap and TMMM)} & \multirow{10}{*}{Telehealth} & \multirow{10}{*}{Yes, evaluated} \\ \cline{1-1}  & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 10: _Cont._discussed the aspects that should be considered to ensure the sustainability of IT systems in healthcare. Examples of these indicators are modifiability, reusability, portability, and supportability of the IT system. [PERSON] et al. [33] developed a conceptual framework that describes how organizational and technical factors influence the sustainability of eHealth implementation. The framework has four sections. These sections are (1) inputs, which contains organizational and technological factors, (2) processes, which deals with activities executed to achieve the desired outputs, (3) outputs, which consists of intention to use, actual use, and user satisfaction, and (4) outcomes, which represents the organizational benefits (i.e., improving the decision-making process and internal communications in the organization). By using a causal loop diagram (CLD), the study presented the dynamic interactions between organizational factors and the technological dimensions. Additionally, the study drew insights from the literature about measuring the technological aspects. For system quality, it could be measured by ease of use, functionality, reliability, flexibility, portability, integration, and importance of the system [42; 43; 44]. The measures of information quality consist of accuracy, timeliness, completeness, relevance, and consistency of information produced by the IT system [42; 43; 44; 45]. Moreover, the service quality measures include up-to-date hardware and software, reliability, responsiveness, assurance, and empathy of technical support [43; 44]. The model proposed by [PERSON] [34] aimed to be used as an evaluation method for developed and under-developed m-health systems to assess the sustained use of them. The model focuses on evaluating aspects related to the user (i.e., patients and physicians) and system design (i.e., hardware and software), each of which are intended to be measured by endorsed standards. The standards are included in Table 10.  
As requirements engineering plays a vital role in software sustainability, [PERSON] et al.