Conducting cost-benefit analyses of architectural attributes such as security has always been difficult, because the benefits are difficult to assess. Specialists usually make security decisions, but program managers are left wondering whether their investment in security is well spent. This paper summarizes the results of using a cost-benefit analysis method called SAEM to compare alternative security designs in a financial and accounting information system. The case study presented in this paper starts with a multi-attribute risk assessment that results in a prioritized list of risks. Security specialists estimate countermeasure benefits and how the organization's risks are reduced. Using SAEM, security design alternatives are compared with the organization's current selection of security technologies to see if a more costeffective solution is possible. The goal of using SAEM is to help information-system stakeholders decide whether their security investment is consistent with the expected risks. IT managers are motivated to minimize security costs but maximize security benefits. Comparing costs among alternative security architectures is significantly easier than comparing benefits, since proven financial analysis tools can more precisely estimate costs. In contrast, benefits are based on uncertain events and imperfect knowledge. Although no one can accurately predict how often an attack will occur and how effectively the security will mitigate the damage, experienced security managers intuitively, and implicitly, estimate the risk and the effectiveness of their risk mitigation strategies. The key to security cost-benefit analyses is to make these intuitions explicit. In addition to the difficulties in estimating risk, comparing alternative designs is challenging because the strength of the design depends on a relaxed adherence to security engineering design principles. For example, security managers usually adopt a precautionary approach to security engineering. Security architectures that have at least one mitigation strategy for each risk are usually preferred to those that leave gaps for rarely expected attacks. Simplistic cost-benefit analyses will have difficulty showing economic justification for rare events. This paper presents the Security Attribute Evaluation Method (SAEM) and the results of using this method to compare alternative security designs for a non-profit organization's financial and accounting system. SAEM is a cost-benefit analysis process for analyzing security design decisions that involves four steps: 1) a security technology benefit assessment, 2) an evaluation of the effect of security technologies in mitigating risks, 3) a coverage assessment and 4) a cost analysis. Steps 3 and 4 can be done in parallel. SAEM relies on a quantitative risk and benefit assessment, in which an analyst or investigator conducts structured interviews of IT and security managers to elicit the initial data. The organization carefully reviews the results of each step before continuing to the next step. If the results do not appear to represent the managers' concerns or experience, then the managers can revise the original input data and assumptions. Risk mitigation strategies include procedures and security technologies. As presented in this paper, SAEM only evaluates the technology design choices, not risk mitigating procedures. Though SAEM could be expanded to include procedures as part of the assessment, these procedures tend to be organization 232 specific and SAEM was initially designed to be used across organizations. Currently, I am developing SAEM in collaboration with security managers of real systems. In addition to the financial and accounting system, I am also using SAEM to evaluate a government e-commerce system and a Department of Defense web-based logistics system. Section 2 assesses the current practice of security architecture development. Section 3 lays the foundation for the Security Attribute Evaluation Method with the multi-attribute risk assessment. The section presents a brief description of the multiattribute risk assessment method, which was used to prioritize the organization's risks based on the security manager's estimate of attack frequencies and potential outcomes. Section 4 describes SAEM and presents an example of how SAEM could be used to select the most cost-effective security technology from among three alternatives. Section 5 shows the results of conducting sensitivity analysis on the initial results and Section 6 reports on the feedback from case study clients. Finally, Section 7 discusses future work.  SAEM depends upon a risk assessment and an initial assessment of security technology effectiveness. These assessments depend upon several assumptions, such as: That the organization has established security policies and procedures that are sufficient and robust for the business operation; That security products have been correctly installed, configured and maintained; That attacks result in predictable outcomes and variances. These assessments will vary among security managers based on their experience, expertise, and information system environment [7] . SAEM provides a framework to analyze how their assumptions affect design decisions and a mechanism to determine how sensitive these decisions are to the assumptions. Although one may argue with the estimations presented in this paper, they do represent an experienced security manager's best estimations.  Develop and
Approaches like Multi-Attribute Risk Assessment (MARA) REF can improve the risk assessment process by considering multi-attributes.
Security attribute evaluation method: a cost-benefit approach
Proceedings of the 24th International Conference on Software Engineering. ICSE 2002
Proceedings of the 24th International Conference on Software Engineering. ICSE 2002
Computer Science