## 6 Results and Discussion
### Results Validation Using Evaluation Metrics  
The total number of RVQA data confidentiality violation detections was evaluated using confidence and sensitivity metrics [54; 55; 56]. Confidence (precision) is the percentage of the detected violations from the overall detection result that are correct (when compared to the known data confidentiality violations). Sensitivity (recall) is the percentage of the detections from the overall known violations that are detected and correct (when compared to the known data confidentiality violations). To use the metrics, the expected total number of the detected data confidentiality violations for the experiment was calculated using Equation (7). In this experiment, x is the agent identification and \(n\) is the number of agents executed in CNS. \[ \begin{split}\text{ExpectedTotalDetectedDCV}\ =\sum\limits_{x=1}^{n} \left(\text{noDisclosedContactx}\ +\ \text{noDisclosedIdentityx}\ +\ \text{noDisclosedLocationAtReak}\ +\ \text{noDisclosedSocialInformationx}\ +\ \text{noDisclosedContentx}\right)\end{split}  \tag{7}\]  
Figure 16 shows the confidence of the RVQA results throughout the 24 h of the CNS execution. From the start, the graph shows a highly increased confidence percentage. The graph then gradually increases and stays constant above 95% as more than 95% of the subsequent detected data confidentiality violations were known violations that were expected to be detected. However, during the classification of violations, RVQA managed to include violations that were not in the list of expected violations. For example, RVQA classified all the disclosed locations as data confidentiality violations, while the list of expected violations only considered location disclosures as data confidentiality violations when the agent identity was also disclosed. The confidence percentage was expected to stay constant. This result, however, can be improved upon if the data confidentiality requirements and checking rules at the AQM definition phase are refined in order for all the data confidentiality detections to become the correct or expected detections (when compared to the known data confidentiality violations). Figure 15: Number of detected data confidentiality violations in the CNS by the RVQA and CNS Searching. Figure 17 shows the sensitivity of the RVQA detection results throughout the 24 h of the CNS execution. From the start, the graph shows very a high sensitivity percentage. The graph then stays constant, very close to 90%. This means that almost 90% of the violations that were expected to be detected had been correctly detected. The sensitivity or recall percentage was expected to stay constant. However, improvement may be made in the future where new data confidentiality requirements and checking rules at the AQM definition phase may be defined in order for other data confidentiality conditions, such as the disclosure of an agent contact, can be checked and the violations that may occur can be detected. The above experiment was repeated and the results at the end of the 24 h of each experiment are presented in Table 4. The table displays the results of the detected data confidentiality violations for the three repeated experiments at all levels of agent data confidentiality. The results show that the RVQA process managed to detect the expected data confidentiality violations at an average confidence between 84% to 90% and average sensitivity in the 79% to 83% range. It also shows that the RVQA process managed to correctly detect data confidentiality violations at an average confidence (precision) of 84% to 90% out of the total number of detections. In other words, the average confidence shows  
Figure 16: Confidence of the data confidentiality violations detected using RVQA. Figure 17: Sensitivity of the data confidentiality violations detected using RVQA. that from 100 detections, 84 to 90 of the detections were correct detections (when checked with known violations that were expected to be detected). Therefore, the confidence of the RVQA was very high (close to 90%).