## 3 Research Execution and Analysis
### Calculation of Usability Metric Weights
#### 3.3.3 Calculation of Weights  
By standardizing the judgment matrix through its eigenvector computation, we progressively determined the relative importance of each component. The weight values accumulated at each hierarchical level were measured in relation to the overarching objective and were calculated progressively from the upper to lower tiers. Within the framework of a layered decision-making process, the weight designated to each level underpins the assessment of the elements' relative significance within that level [88]. The procedure for computing the sorting method for individual levels is outlined as follows:  
1. Calculate the product of each row's indicators in the judgment matrix \(M_{i}\). \(m\) is the total number of indicators in the judgment matrix. \[M_{i}=\prod\
olimits_{j=1}^{m}a_{ij}(j=1,2,\cdots,m)\] (2)
2. Calculate the nth root of \(M_{i}\). \[W_{i}=\sqrt[n]{M_{i}}\;(i=1,2,\cdots,n\) )
3. Normalize \(W_{i}\) to obtain the eigenvector \(\omega_{i}\). \[\omega_{i}=\frac{W_{i}}{\sum_{j=1}^{m}W_{j}}(j=1,2,\cdots,m)\] (4)
4. The formula for the maximum value of the judgment matrix is as follows: \begin{table}
\begin{tabular}{l l} \hline \hline \multicolumn{3}{c}{**Saaty’s 1-9 Scale Assignment Method**} \\ \hline \hline
**Scale** & **Meaning** \\ \hline \hline
1 & Indicators \(i\) and \(j\) are of equal importance. \\ \hline
3 & Indicator \(i\) is slightly more important than indicator \(j\). \\ \hline
5 & Indicator \(i\) is moderately more important than indicator \(j\). \\ \hline
7 & Indicator \(i\) is strongly more important than indicator \(j\). \\ \hline
9 & Indicator \(i\) is absolutely more important than indicator \(j\). \\ \hline
2, 4, 6, 8 & The importance of the indicators falls between the above scales. \\ \hline \hline \end{tabular} \
\begin{tabular}{l l l l} Reciprocal &
\begin{tabular}{l} If the comparison between factors \(i\) and \(j\) results in the judgment matrix \\ entry \(C_{ij}\), then the comparison of factor \(j\) to \(i\) is given as \(C_{ij}=1/C_{ij}\). \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Judgment matrix scoring criteria. \[\lambda_{max}=\sum_{i=1}^{n}\frac{\left[A\omega\right]_{i}}{n\omega_{i}} \tag{5}\]  
Consistency Test  
The Consistency Ratio (CR) is defined as the ratio of the Consistency Index (CI) of the judgment matrix to the Random Consistency Index (RI). The judgment matrix is deemed consistent if the CR is less than 0.1. If the CR exceeds 0.1, the judgment matrix needs to be reconstructed [89]. The formula for calculating the CR is as follows:  
\[\text{CI}=\frac{\lambda_{\text{max}}-n}{n-1} \tag{6}\]  
\[\text{CR}=\frac{\text{CI}}{\text{RI}} \tag{7}\]  
where the RI value in the equation is based on Table 9.  
The Analytic Hierarchy Process (AHP) utilizes Saaty's 1-9 scale for pairwise comparisons. Accordingly, we first organize the data scored by 15 experts using Saaty's rating scale into Tables 10-15. The evaluation method can be illustrated using the primary indicators B1 and B2 from Table 10. The ratio of the B1 indicator on the vertical axis to the B1 indicator on the horizontal axis is 1 since they are the same indicator. The ratio of the B1 indicator on the vertical axis to the B2 indicator on the horizontal axis is 7, indicating that B1 is significantly more important when compared to B2. Conversely, the ratio of the B2 indicator on the vertical axis to the B1 indicator on the horizontal axis is 1/7, which also indicates that B2 is significantly less important. Similarly, in pairwise comparisons of B2 and B3 indicators, the ratio of the B2 indicator on the vertical axis to the B3 indicator on the horizontal axis is 1/5, showing that B3 is more important than B2. By analogy, all pairwise comparison data in the table are derived from experts' evaluations of the importance of each pair of indicators using Saaty's rating scale. Finally, based on the experts' scores, Formulas (2)-(7) are applied to determine the weights at each level, which are then annotated in Tables 10-15.