Vulnerability identification errors in security risk assessments

At present, companies rely on information technology systems to achieve their business objectives, making them vulnerable to cybersecurity threats. Information security risk assessments help organisations to identify their risks and vulnerabilities. An accurate identification of risks and vulnerabilities is a challenge, because the input data is uncertain. So-called ’vulnerability identification errors‘ can occur if false positive vulnerabilities are identified, or if vulnerabilities remain unidentified (false negatives). ‘Accurate identification’ in this context means that all vulnerabilities identified do indeed pose a risk of a security breach for the organisation. An experiment performed with German IT security professionals in 2011 confirmed that vulnerability identification errors do occur in practice. In particular, false positive vulnerabilities were identified by participants. In information security (IS) risk assessments, security experts analyze the organisation’s assets in order to identify vulnerabilities. Methods such as brainstorming, checklists, scenario-analysis, impact-analysis, and cause-analysis (ISO, 2009b) are used to identify vulnerabilities. These methods use uncertain input data for vulnerability identification, because the probabilities, effects and losses of vulnerabilities cannot be determined exactly (Fenz and Ekelhart, 2011). Furthermore, business security needs are not considered properly; the security checklists and standards used to identify vulnerabilities do not consider company-specific security requirements (Siponen and Willison, 2009). In addition, the intentional behaviour of an attacker when exploiting vulnerabilities for malicious purposes further increases the uncertainty, because predicting human behaviour is not just about existing vulnerabilities and their consequences (Pieters and Consoli, 2009), rather than preparing for future attacks. As a result, current approaches determine risks and vulnerabilities under a high degree of uncertainty, which can lead to errors. This thesis proposes an approach to resolve vulnerability identification errors using security requirements and business process models. Security requirements represent the business security needs and determine whether any given vulnerability is a security risk for the business. Information assets’ security requirements are evaluated in the context of the business process model, in order to determine whether security functions are implemented and operating correctly. Systems, personnel and physical parts of business processes, as well as IT processes, are considered in the security requirement evaluation, and this approach is validated in three steps. Firstly, the systematic procedure is compared to two best-practice approaches. Secondly, the risk result accuracy is compared to a best-practice risk-assessment approach, as applied to several real-world examples within an insurance company. Thirdly, the capability to determine risk more accurately by using business processes and security requirements is tested in a quasi-experiment, using security professionals. This thesis demonstrates that risk assessment methods can benefit from explicit evaluation of security requirements in the business context during risk identification, in order to resolve vulnerability identification errors and to provide a criterion for security.