# Developing Security Assurance Metrics to Support Quantitative Security Assurance Evaluation
## 4 Security Assurance Analytics
### Security Assurance Scoreboard  
A scorecard is a report technique that allows management to measure the achievement of current activities of the organization in comparison to planned goals or outcomes [34], which includes a set of statistical records and meanwhile employs a top-down approach to present the indicators. Figures 3-5 demonstrate exemplary assurance scoreboards filled with our proposed assurance metrics. In this study, an evaluation result of an open-source web-based issue management system is presented. This system allows project members to document and track project issues as well as to generate management reports. For the SA evaluation to work, we leveraged OWASP as the knowledge source to prepare the content of assurance components in the domain of web applications. Three OWASP project materials were chosen: OWASP ASVS, OWASP Top 10, and OWASP Web Security Testing Guide (WSTG) [35]. The first material was used to construct a set of security requirements, while the last two were synthesized for the vulnerability assessment. As a result, a total of eight security requirements and eight vulnerabilities are considered crucial for evaluating the assurance target. Figure 4: Security assurance scoreboard—security requirement metrics. Figure 3: Security assurance scoreboard—top level. In Figure 3, the \"top-level\" scoreboard structure is presented, which comprises assurance metrics showing the summarized SA evaluation result of the assurance target, including the overall assurance score, the assurance level, and the assurance metrics in the perspective of the security requirement and the vulnerability. These data provide a \"panoptic view\" by aggregating the metrics from both sets of assurance perspectives (security requirements and vulnerabilities). For a better comparison of metrics, the corresponding maximum and actual values are listed side-by-side. From Figure 3, we can see that the security requirement perspective has a major role in the overall SA evaluation since it achieves up to 73.76% of the assurance score, while the vulnerability perspective, on the other hand, contributes only 26.24%. Furthermore, in terms of performance, the security requirements generally gain a worse score than vulnerabilities (76.20% vs. 86.84%). Thus, it is suggested that security requirements should be treated as higher priorities while taking improvement actions. Figures 4 and 5 are two drill-down scoreboards providing the \"next-level\" details of security requirement metrics and vulnerability metrics. The two tables permit metrics to be monitored through the level of assurance criteria, which allows an effective and \"tidy\" collection of the stakeholders' views of the areas of interest. In that table, rows identify the criteria, while the columns identify the areas of interest and the key assurance metrics. It is noted that, in Figure 4, the weight factors for all security requirement criteria are estimated by security and domain experts using a subjective weighting approach. As we can see, among the eight security requirement criteria, \"Access Control\" has the highest actual impact on security assurance score calculation (i.e., 9.25%) as well as the best performance (85%). On the other hand, \"Stored Cryptography\" is the worst assurance criterion since both the performance and the impact on SAS are the lowest (57.6% and 4.18%, respectively). According to the vulnerability metrics shown in Figure 5, the criterion \"Vulnerable and Outdated Components\" has a maximum impact on security assurance score, meanwhile, it reaches an outstanding performance (100%). As the poor performance of \"Cryptography\" is identified in the security requirement perspective, the corresponding weakness in terms of the \"Cryptographic Failures\" is also disclosed in the vulnerability perspective (performance: 73%).