# Helping CNAs Generate CVSS Scores Faster and More Confidently Using XAI
## 3 Related Works
### Adoption of Deep Learning in CVSS Prediction  
Starting in 2020, authors started using Deep Learning, a subfield of Machine Learning that uses Artificial Neural Networks to model and solve complex problems. This switch in technology has further improved results. Then, in 2017, [PERSON] et al. [7] experimented with a Convolutional Neural Network, a type of Deep Learning architecture known for its immense success in image classification that also works well on one-dimensional data like text. They used it in conjunction with an SVM to achieve impressive results on version 3.0 of the CVE data. For example, [PERSON] et al. [9] were the first to use BERT [41], which stands for Bidirectional Encoder Representations from Transformers. BERT is a pretrained natural language processing model developed by Google based on the Transformer architecture. It is trained on a large corpus of text data to learn the underlying patterns and relationships between words \begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**Technique** & **Method Type** & **Example Format** & **Data Type** & **Compatibility** & **Algo. Type** & **Granularity** \\ \hline Anchors [31] & Post hoc & Text rules & Any & Agnostic & Proximity & Local \\  & & & & & training & \\ \hline SHAP [32] & Post hoc & Image & Any & Agnostic & Perturbations & Local \\ \hline LIG [33] & Post hoc & Image & Image & Neural networks & Gradients & Local \\ \hline LIME [34] & Post hoc & Image & Any & Agnostic & Proximity & Local \\  & & & & & training & \\ \hline XGBoost [35] & Ante hoc & Vectors & Tabular & Tree & Interpretable & Global \\ \hline LRP [36] & Post hoc & Image & Any & Neural networks & Gradients & Local \\ \hline Occlusion Sensitivity [37] & Post hoc & Image & Image & Agnostic & Perturbations & Local \\ \hline Saliency Maps [38] & Post hoc & Image & Image & Neural networks & Gradients & Local \\ \hline PDP [39] & Ante hoc & Image & Tabular & Agnostic & Perturbations & Global \\ \hline Linear Regression [40] & Ante hoc & Vectors & Tabular & Linear & Interpretable & Global \\ \hline \hline \end{tabular}
\end{table} 
Table 2: XAI methods survey. and sentences. BERT is capable of understanding the context and nuances of language and can be fine-tuned for various natural language processing tasks. In their work, [PERSON] et al. [9] used BERT in combination with LSTM (Long Short-Term Memory), which is a type of Recurrent Neural Network architecture widely used for sequence modeling and prediction tasks. They proposed a framework called ExBERT, which utilizes BERT and transfer learning to learn sentence-level semantic features from vulnerability descriptions and capture long dependencies using an LSTM classifier. Additionally, they incorporated the Exploit Database as an additional source of information. Similarly, [PERSON] and [PERSON] [10] introduced CVSS-BERT, a tool that employs multiple BERT classifiers, with each classifier dedicated to predicting a specific metric of the CVSS vector. This approach achieves high accuracy in predicting the values of individual metrics. Unlike previous works that fine-tuned a single BERT model on the entire dataset, CVSS-BERT fine-tunes each model independently. This means that the error function focuses solely on the success of one metric at a time rather than the average performance across all metrics. Consequently, the predicted severity scores closely align with the severity scores provided by human experts. To reduce computational costs and promote wider adoption, the authors utilized BERT-small, a smaller version of BERT with fewer layers and parameters. [PERSON] et al. [11], similar to [PERSON] et al. [9], utilized a single model for all metrics and incorporated additional data from Open-Source Intelligence Information Sources such as news articles and security blogs. However, they employed DistilBERT, which is a smaller and faster version of the BERT language model.