## 5 Evaluation Approach
### Analysis of Feature Models and Diagnosis of Configurations  
The analysis of the previous models is focused on validating them and computing the available configurations. The results are shown in Table 3. The models are valid, therefore, at least one valid configuration would be obtained from each model. All the models have been validated and it would be reached without the automatic analysis provided by CyberSPL. Regarding the configurations, the number of possible configurations for the _Apache_ model is 96, for the _SSL/TLS_ model is 1482, for the _Android_ is 262,144, and for _Linux_ is 56,448. Even limiting the configurations to a few parameters, the complexity of the configuration continues to be very high. The high number of configurations of _Android_ and _Linux_ models with regard to the _Apache_ and _SSL/TLS_ models is derived from the type of relations and cross-relations. Thus, a high number of cross-relations will limit the number of solutions that satisfy the relations of the model. Despite this, the configurability problem is intractable whether it would be done manually by a human. The complexity increases by combining the systems, since the number of configurations would increase significantly because of the combinatorial explosion in the number of possible configurations. Figure 11: Security Linux Kernel Feature Model. Figure 12: Security Android Feature Model. Thus, there is a total number of 320,650 configurations among the four models. The automatic resolution of problems by means of CSP solver helps to deal with this complexity. The resolution of constraint-based problems is a well-known problem in the literature [50]. However, this complexity can be dealt with CSP solvers as shown in previous works, [16; 57]. In both works, the complexity is studied in terms of performance and scalability of constraint-based problems, and we demonstrated remarkable results from the usability perspective response time in the resolutions. One of the advantages of CyberSPL is the possibility of diagnosing configurations as shown in Figure 13. The idea is to check if a configuration conforms to the cybersecurity policy represented in the feature models. In case of invalid configurations, CyberSPL is able to establish the diagnosis, that is, to determine what are the possible failures in them. In Figure 13, a diagnosis is depicted in the console that recommends to the user to select _TLSv1.2_ or _TLSv1.3_ and deselect _SSLv3_. For each model six configurations have been tested, two valid and four invalid. The invalid configurations have been diagnosed and the explanation of the diagnosis is given. The cases of _Apache_ and _SSL_ are shown in Tables 4 and 5, respectively. Obtaining the diagnosis is more than determining whether a configuration is valid or not, it is an explanation about why a configuration is invalid according to the established cybersecurity policies. In particular, the diagnosis indicates the features that must be selected and deselected. Following the definition of diagnosis, _Select_ an feature means that it had a _false_ value in C (invalid configuration) and in C' (valid configuration) it would have the \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Feature Model** & _Apache_ & _SSL/TLS_ & _Android_ & _Linux_ \\ \hline Number of features & 27 & 48 & 24 & 20 \\ Mandatory & 10 & 8 & 0 & 0 \\ Optional & 3 & 0 & 5 & 18 \\ OR & 1 & 1 & 5 & 0 \\ XOR & 3 & 9 & 0 & 0 \\ Cross-Relations & 2 & 12 & 0 & 0 \\ \hline Valid & ✓ & ✓ & ✓ & ✓ \\ Number of configurations & 576 & 1482 & 262,144 & 56,448 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Analysis of the feature models. Figure 13: Diagnosis results in CyberSPL. true_ value. On the other hand, _Deselect_ means that in C (invalid configuration) the concrete feature have a _true_ value and in C' (valid configuration) it would have a _false_ value.