# The Cybersecurity Focus Area Maturity (CYSFAM) Model
## 6 Evaluation
### Expert Evaluation and Results  
In a mixed-methods qualitative research engagement, expert interviews are a very suitable and fruitful way to establish construct validity as discussed by [60]. Therefore, it was decided to evaluate CYSFAM by means of expert evaluations. The selection criteria for the experts were as follows:  
* The interviewee is experienced in the information or cybersecurity domain--a minimum of 5 years of experience is required.
* The interviewee is capable of thinking outside of the context of the case study company.
* The interviewee can prove their knowledge by providing Certified Information Systems Security Professional (CISSP) certification [61].  
Two suitable domain experts were approached to participate in the evaluation.  
The first interviewee was an IT security expert who had 15 years of experience and was working at the case study company at the time of the interview was conducted. The second interviewee was an external IT security consultant who was specialized in security in large organizations. Both interviewees had CISSP certification. CYSFAM and the methods applied to develop the CYSFAM were introduced to the interviewees. The interviewees evaluated CYSFAM in separate sessions without the researchers' intervention. Finally, they provided the researchers with their evaluation results.  
Interviews with the experts resulted in the following most important remarks:  
1. CYSFAM entirely lacks a module on security in Cloud Computing, which is one of the most prevalent security concerns for now and in the upcoming years. That certainly needs to be included in the final model.
2. The scoring method is hard to grasp and, in its rigidness, it is somewhat harsh. However, compliance agencies and auditors are usually also harsh; thus, that warrants such a setup.
3. CYSFAM is very control-based. It would be an idea to introduce two dimensions per focus area: control-based and process-based (or process maturity).
4. CYSFAM contains quite some jargon. That is not necessarily an issue, but you need to be clear about the intended target audience of CYSFAM (security experts, rather than management--upper level or not).
5. There are a number of capabilities that could be better rephrased in CYSFAM.  
The authors addressed these remarks as follows: for remark #1, cloud computing was investigated in depth with regard to the security-related capabilities and controls that are in place. The results of this investigation showed that, considering the range of this topic, a cloud security maturity model could very well be an extensive maturity model on its own. Recently, a study on cloud computing security maturity modelling addressed this gap in the literature [62]. Since cloud security computing was considered as reasonable to be a separate maturity model, it was not incorporated as a focus area in CYSFAM. For remarks #2 and #3, there were obvious limitations (lack of knowledge, resources) to not include them in this version of CYSFAM. They do, however, form important challenges for future research engagements. Remark #4 is inherent to the design and scope of this study; cybersecurity is a domain that is largely embraced by technical specialists, rather than management and directors. The improvements proposed along with remark #5 were incorporated in CYSFAM. The authors went through six improvement proposals of the experts-three by each expert- and a common one proposed by both experts. All seven improvements were reflected in the capabilities.