# Helping CNAs Generate CVSS Scores Faster and More Confidently Using XAI
## 1 Introduction
### Objectives  
In this paper, our objective is to lower the time taken by analysts to generate scores while maintaining a high standard of quality assurance by leveraging Machine Learning (ML) and especially Explainable AI (XAI). While ML has already been used for automatically generating CVSS scores [2; 3; 4; 5; 6; 7; 8; 9; 10; 11], the focus has mainly been on the creation of models that perform well on the task, not on explaining the underlying reasoning behind the decisions. Given the potential repercussions of scoring errors on the organizations targeted by vulnerabilities, analysts must exercise caution to safeguard their company's reputation. That is one reason why well-performing black-box ML models cannot be used for this task. Another reason is the strict regulations for Europe-based and USA-based companies:  
* Europe: Since May 2018, companies and organizations in Europe have been required to provide justifications for decisions upon request, as mandated by the General Data Protection Regulation (GDPR) [12], a regulation in EU law on data protection and privacy for individuals within the European Union (EU) and the European Economic Area (EEA). * USA: Simultaneously, in the United States, the Defense Advanced Research Projects Agency (DARPA) [13] launched a multi-year investment of USD 2 billion in artificial intelligence research in 2016, focusing on XAI to develop AI systems capable of explaining their reasoning and decision-making processes to human users. As such, building systems that explain their decision-making processes is as important as building high-performance ones. Our work focuses on the explanation part by leveraging Explainable AI (XAI), a science that makes black-box models transparent by providing understandable explanations for their decision making. This justification will then assist the analysts in their decision making and will be available upon request. To achieve that, we proposed several explanation methods that lessen the analytical burden on the analysts and generate explanations that are comprehensive (cover all the cases), contrastive (allow comparison between classes), and can be easily contested (based on transparent criteria). We also show that Deep Learning models are not necessary for this task by achieving results on par with DL techniques using only Machine Learning, which allows easier and more cost-effective deployment, maintenance, and retraining for the company. Figure 2: Vulnerability severities of the past 7 years. Our contributions are as follows:  
1. Comprehensive and contrastive global explanations: Our multi-class global explanations cover all classes in a way that allows analysts to show how close each vulnerability was to being classified as another class, reinforcing transparency and allowing deeper inspection and anomaly detection. 2. Novel local text explanation method using SHAP: We come up with our own local explanation for multi-class text classification using raw Shapley values. 3. Addressing SHAP Limitations in Text Data Interpretation: We draw attention to a challenge associated with utilizing SHAP for deriving word importance in text data interpretation. Furthermore, we introduce an initial conceptual approach to tackle this issue, aiming to enhance the reliability and precision of word importance assessments. 4. Simple vulnerability resolution rule: We propose a simple IF rule that can be used to deal with a subset of the vulnerabilities, along with empirical results to prove its effectiveness. 5. State-of-the-art modeling results using ML: we show that excellent results can be achieved through XGBoost, an interpretable and fast ML model, instead of complex and expensive Neural Networks. 6. Open-source dataset Our publicly shared dataset is the biggest as of this paper's submission date in terms of the number of vulnerabilities and the feature columns, transformed into an easy-to-use CSV. 7. Open-source code: All codes for data collection, data preprocessing, modeling, and explainability are publicly shared.