# Helping CNAs Generate CVSS Scores Faster and More Confidently Using XAI
## 6 Explainability
### Utilizing Raw Shapley Values for Crafting Local Multi-Class Explanations
Utilize the same preprocessing technique employed for the ML model to extract pertinent words from the CVE description. 2. Compute the cumulative SHAP score for each word in every class. 3. Sum the scores for each class. 4. Identify and highlight the class with the highest score sum. 5. Compare the highest class with the class from the model classification
6. Introduce a check for ambiguity if the class scores are close (within T), prompting the analyst to verify. 7. Transform the interim results into a picture. 8. Return the picture. The final output for this method for an example CVE can be seen in Figure 16. The inclusion of the model confidence score serves the purpose of providing additional insight \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Class** & **Row** & **Word 1** & **Word 2** \\ \hline LOCAL & 12 & \(-0.0009383615\) & \(-8.516443\times 10^{-5}\) \\ \hline LOCAL & 193 & \(-0.00095072796\) & \(-8.516443\times 10^{-5}\) \\ \hline PHYSICAL & 134 & 0 & 0 \\ \hline PHYSICAL & 1849 & \(-0.00000432443\) & \(0.00000034234\) \\ \hline NETWORK & 245 & 0 & \(-9.46\times 10^{-11}\) \\ \hline NETWORK & 9452 & \(0.000001321\) & \(-0.0000043983\) \\ \hline ADJACENT NET. & 3255 & \(0.0000004322\) & 0 \\ \hline ADJACENT NET. & 234 & 0 & \(0.000000428\) \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Flattened version of the SHAP tensor values. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Word** & **Class** & **Score** \\ \hline https & Local & 1.713 \\ \hline connections & Local & 16.403 \\ \hline action & Local & 2.0306 \\ \hline enabled & Network & 47.195 \\ \hline workaround & Network & 24.982 \\ \hline
160 & Network & 1.626 \\ \hline lan & Adjacent network & 279.997 \\ \hline received & Adjacent network & 8.344 \\ \hline ethernet & Adjacent network & 14.587 \\ \hline causes & Physical & 8.748 \\ \hline phone & Physical & 33.111 \\ \hline uefi & Physical & 4.12 \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Random words’ affinity sum sample. into the level of trust we can place in the model, especially in situations where uncertainty exists. It also acts as a valuable tool for detecting instances where the model exhibits false confidence, a potential indicator of data bias or skewness that warrants further investigation. Analysts can benefit from this visual explanation in the following ways:  
1. Guided analysis: This explanation guides analysts through the entire pipeline, allowing them to identify issues and question the decision. For instance, when reviewed by a network specialist, they can discern its coherence and determine whether any adjustments are necessary. 2. Pattern shifts: Over time, analysts can monitor real-world changes by observing shifts in word importance across different classes. 3. Close collaboration: Analysts actively participate in model retraining by providing insights on whether specific words should be included or excluded after preprocessing. This guidance aids ML teams in making informed decisions for future updates. 4. Transparency and comprehensiveness: Analysts can thoroughly examine the impact of each word on each class and make comparisons. This enables them to identify any biases that may need to be addressed or unlearned. 5. Humans in the loop: In the case of ambiguity, the analyst's expertise is needed, fostering a sense of connection to the project and building trust and confidence with the AI. To enhance this approach, considerations for improvement include aiding analysts in selecting the threshold T through an algorithmic analysis conducted beforehand and incorporating the model confidence score for each potential class to increase transparency, facilitating more informed decision making by analysts as needed.