# Developing Security Assurance Metrics to Support Quantitative Security Assurance Evaluation
## 3 Our Methodological Approach
### Assurance Metrics Calculation: The Core Concept
#### 3.2.1 Assurance Element Evaluation Phase  
The first phase of assurance evaluation is responsible for the assessment of the assurance elements of the SA evaluation model, from the quantification of the corresponding assurance conditions. In our test-based methodology, each assurance condition is mapped to one test case to decide fulfillment scores (for SRD) or existence scores (for VUD). For SRD, results for test cases are primarily pass or fail, where a pass indicates that the corresponding SRD is \"Fully fulfilled\" (fulfillment score = 1), while a failure of a test case means that the SRD is \"Not fulfilled\" (fulfillment score = 0). However, in some test cases, the result can be considered a \"Partial fulfillment'. Partial fulfillment means that the actual result matches its expected result, however, there are more rigorous criteria/specifications needing to be met in order to strongly claim the full fulfillment. In addition, an unnecessary (or superfluous) exception/message that is caught during the test-case execution can be also treated as a partial fulfillment [28, 29]. Such a test execution state is usually applied in the context of manual testing, heavily reliant on the tester's judgment [30]. For example, it is assumed that the SHA-1 encryption algorithm is found in testing the SRD \"The system stores account password in approved encrypted formats\". In this case, even though there is evidence showing the password is encrypted, we see this test case to be a partial pass, as the SHA-1 is not considered a strong-enough password encryption function [31]. Therefore, the assurance score of the SRD is assigned a value of 0.5, indicating \"Partially fulfilled\". Similarly, the existence score for VUD has two value options, where 0 means no vulnerability indicated by the test results, and 1 represents the existence of the vulnerability.  
The scores for SRE and VUE are calculated separately. For an SRE, once the fulfillment scores are decided in all associated SRDs, its score can be calculated. We define a metric \(ActSRD\) as a measurement to reflect the actual (calculated) score of SRE. The value of \(ActSRE\) is obtained by averaging the fulfillment scores of the related SRD. Since the SRDs  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Symbol** & **Description** \\ \hline AT & Assurance target \\ SRP & Security requirement perspective \\ VUP & Vulnerability perspective \\ SRC & Security requirement criteria \\ VUC & Vulnerability criteria \\ SRE & Security requirement element \\ VUE & Vulnerability element \\ SRD & Security requirement condition \\ VUD & Vulnerability condition \\ \hline \hline \end{tabular}
\end{table}
Table 4: Symbols used in the assurance evaluation.  
Figure 2: Sample hierarchical structure of the security assurance evaluation.  
we add together are similar ones, by using the \"Average\" function, we can consider all the relevant items to derive a representative score of the whole dataset. Additionally, the assurance conditions are designed in such a way that each condition will cover one perspective of the assurance element. Failing the whole element if one condition fails is not fair for the rest of the conditions. The following formula represents the calculation of the \(i\)-th SRE score (represented as \(ActSREi\)):  
\[ActSRE_{i}=\frac{\sum_{j=1}^{n}ActSRD_{ij}}{n},\ \forall ActSRD\in\ \{0,\ 0.5,\ 1\}, \tag{1}\]  
where:  
\(ActSRDij\): the actual (fulfillment) score of the \(j\)-th SRD associated with the \(i\)-th SRE;  
\(n\): the number of SRDs associated with the \(i\)-th SRE.  
Similarly, the formula used for calculating the actual VUE score (\(ActVUE\)) is defined as the average of the corresponding VUD existence score, represented below:  
\[ActVUE_{i}=\frac{\sum_{j=1}^{n}ActVUD_{ij}}{n},\ \forall ActVUD_{ij}\ \in\ \{0,\ 1\}, \tag{2}\]  
where:  
\(ActVUD_{ij}\): the existence score of the \(j\)-th VUD associated with the \(i\)-th VUC;  
\(n\): the number of VUDs associated with the \(i\)-th VUE.