## 5 Result and Discussion
### Evaluation Metrics  
Performance metrics were used to evaluate the output performance of the model. Confusion matrix, accuracy, recall, precision and F1-score metrics were preferred to evaluate the performance of the proposed model. The purpose of using the preferred metrics in the model is briefly explained below.  
Accuracy: This is a metric that produced a percentage of how many correct results the proposed model produced for the whole dataset (Equation (8)).  
Recall: This permitted the determination of how many of the outputs that the proposed model should have positively predicted were correctly predicted (Equation (10)).  
Precision: This was used to measure the performance of the accuracy of the proposed model's positively correct predictions (Equation (11)).  
F1-score: Although precision and accuracy metrics were used for the performance of the proposed model, it was also verified using this metric, which gave a more precise result and was a combination of other metrics (Equation (12)). The F1-score provided more sensitive evaluations for an accurate evaluation, especially if the distribution of some vulnerability clusters in the dataset was not equal.  
[PERSON]'s correlation coefficient (MCC): This produced a value between \(-1\) and \(1\) to evaluate the performance of the model in multiple vulnerability classifications within the VUMREM model. The closer this value was to \(1\), the higher the interpreted classification success (Equation (13)).  
\[\gamma_{accuracy}=\frac{TP+TN}{TP+FN+FP+TN} \tag{9}\]  
\[\gamma_{Precision}=\frac{TP}{TP+FP} \tag{10}\]  
\[\gamma_{recall}=\frac{TP}{TP+FN} \tag{11}\]  
\[\gamma_{f1 score}=2\times\frac{\gamma^{Precision}\times\gamma_{recall}}{\gamma^ {Precision}+\gamma_{recall}} \tag{12}\]  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Hyperparameter** & **Value** \\ \hline NB & smoothing & 0.5 \\ \hline \multirow{2}{*}{k-NN} & neighbors & cosine similarity \\  & dm (distance measure) & 12 \\ \hline \multirow{8}{*}{SVM} & svm type & nu-SVC \\  & kernel type & rbf (radial basis function) \\ \cline{1-1}  & gamma & 0.1 \\ \cline{1-1}  & nu & 0.5 \\ \cline{1-1}  & cache size & 100 \\ \cline{1-1}  & \(\varepsilon\) (epsilon) & 0.001 \\ \cline{1-1}  & weight & 1 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Hyperparameters to train bag-of-words-based models.  
\[MCC_{multiclass}=\frac{c.s-\sum_{k}^{k}p_{k}.t_{k}}{\sqrt{(s^{2}-\sum_{k}^{k}p_{k}^{2 })(s^{2}-\sum_{k}^{k}t_{k}^{2})}} \tag{13}\]  
The explanation of the abbreviations of the metrics used in Equations (1)-(5) are as follows:  
* True Positive (TP): When data were correctly predicted to be vulnerable or non-vulnerable.
* True Negative (TN): The prediction that a non-vulnerable code block was not vulnerable.
* False Positive (FP): The prediction that a non-vulnerable code block was vulnerable.
* False Negative (FN): A prediction state that indicated a vulnerable code block to be not vulnerable.
* k: The number of classes in the model labelled as vulnerable and non-vulnerable.
* \(x=\sum_{k}^{K}X_{KK}\): The number of correct predictions of those labelled as vulnerable and not vulnerable.
* \(s=\sum_{k}^{K}\sum_{j}^{K}X_{ij}\): The total dataset was the number.
* \(p_{k}=\sum_{k}^{K}X_{ki}\): The number of predictions of k classes in the dataset.
* \(t_{k}=\sum_{i}^{k}X_{ki}\): The number of correct recognitions of class k in the dataset.