# Toward a Comprehensive Understanding and Evaluation of the Sustainability of E-Health Solutions
## 6 E-Health Sustainability Assessment Method
### Procedures of Sustainability Assessment
#### 6.2.2 Assessment Steps  
The e-health sustainability model consists of three levels, as demonstrated in Figure 4. The assessment tools are applied to the elements (third level) of the sustainability model. The assessment results of the elements will be reflected in the aspects' fulfillment of achieving sustainability, which in the end contributes to predicting the overall sustainability of the e-health solution. Figure 7 shows the assessment steps, which are explained in the following subsections. Figure 6: Procedures of sustainability assessment. Step 1: Dimensions' Weights  
As the dimensions represent the first level of the e-health sustainability model, they will be given weights based on the assessors' judgment. The given weights reflect the importance of each dimension according to the context in which e-solution is going to be used [25]. For example, if the proposed system is going to be used by healthcare providers, the social dimension becomes very important compared to a situation where the system is going to be used by the IT department only. The weight of each dimension must be between 0 and 1, in which the sum weights of all dimensions must add up to one. Because the assessment will be conducted by a team, it is recommended to use the group analytic hierarchy process (AHP) method. This method assists the team to decide on the weights that will be assigned to each dimension in the sustainability model [67]. Step 2: Selection of Sustainability Elements  
Software products vary and there is no unified context of use for all of them [49]. Therefore, the elements under each aspect will be selected based on their applicability to the e-health solution's context. The selection of aspects and elements is carried out by the assessment team and must be based on the applicability, not the availability, of the aspects. For example, if the e-health solution should be used with special hardware components, then the hardware quality aspect in the technology dimension is applicable to be a part of the evaluation, even if the required information about the hardware quality is not available. In the case where hardware quality is required but the relevant data is not available, the hardware quality aspect will give a very low score that means its assessment is not fulfilled. To differentiate between the assessment of an aspect with a low score that results from unavailable data and low scores resulting from available data, when experts decide that an aspect is relevant, they also have to indicate whether data is going to be available for the evaluation or not. Hence, if data can be collected or it is already available, the experts can proceed with filling out the survey or the checklist; otherwise, the aspect will be assigned the lowest score until data becomes available and then assessment is run again. After selecting the applicable elements, the checklists and surveys will be completed by the assessment team and the overall sustainability level will be calculated based on the following steps and formulas. Figure 7: Sustainability assessment steps. Steps 3-7: Calculating Evaluation Score  
As each aspect has a set of five-point Likert scale questions, to determine the score of each aspect, the following equation is applied:  
\[\underline{\sum\text{Evaluation value of each question of elements of the selected aspects}}\]  
\[\#\text{of total questions in the aspect}\times 5\]  
This step is applied to all aspects within the dimension. Lastly, the overall score of the e-health solution sustainability is the aggregation of the weighted scores of the dimensions, which is calculated by the below equation. It is worth mentioning that this assessment method does not include any mathematical model. The equations used are for weighting the scores of each aspect and dimension. \[\sum\text{Score}(D_{n})\times\text{Weight}(D_{n})\]  
Step 8: Mapping Evaluation Score to Sustainability Level  
The sustainability levels indicate the prediction of the extent to which the e-health solution is considered to be sustainable.