## 3 Sharing ML Models as Cyber Threat Intelligence
### Sharing ML Models as Threat Intelligence
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Field** & **Value** & **Field** & **Value** & **Field** & **Value** & **Field** & **Value** \\ \hline _id.orig\_p_ & 54492 & _duration_ & 0.102117 & _missed\_bytes_ & 0 & _resp\_ip\_bytes_ & 1147 \\ _id.resp\_p_ & 80 & _orig\_bytes_ & 274 & _orig\_pkts_ & 5 & \\ _proto_ & 0 & _resp\_bytes_ & 935 & _orig\_ip\_bytes_ & 486 & \\ _service_ & 0 & _conn\_state_ & 0 & _resp\_pkts_ & 5 & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Feature vector of a distributed denial of service (DDoS) connection correctly classified as malicious. Figure 4: Confusion matrix for the decision tree model. these new PCAP files through Zeek to produce log entries in 'conn.log', the decision tree correctly classified the PCAP with the unmodified HTTP response as malicious, but indeed misclassified the adversarial example with the modified HTTP payload as benign. In practice, after one security analyst shared the ML model through MISP, a second could contribute to this threat intelligence by sharing this particular PCAP file as an adversarial example using the MISP object template we proposed earlier in Listing 4. #### 3.3.3 Sharing a Random Forest Model for Improved Classification

Continuing the story of sharing ML-based threat intelligence, and given the rather trivial decision tree example, a security analyst decides to try a more sophisticated random forest model. Hence, we trained a 'RandomForestClassifier' model with 20 decision trees and an unbounded depth on the same data. The file size of this model is significantly larger, i.e., 171,202 bytes. The confusion matrix of this ML model is shown in Figure 5. With fewer false positives and false negatives, the random forest model performs even better than the decision tree, with an average 10-fold cross-validation score of 0.99999. The security analyst may contribute to the decision tree threat intelligence in MISP by also sharing their random forest model. After all, it has a better classification accuracy. However, the adversarial DDoS examples obtained through modifying the HTTP response payload are still misclassified as benign. Hence, slightly enlarging the payload with a few bytes will also fool the random forest model. #### 3.3.4 Detecting Attacks as Outliers with One Class Support Vector Machine Models

Another security analyst attempts to train a model that learns how benign traffic looks like in an unsupervised manner, so that any deviation could be considered as anomalous or malicious. Thus, we trained a 'OneClassSVM' model in an unsupervised manner on a subset of benign feature vectors only. This model uses an Radial Basis Function (RBF) kernel with the \(\
u\) parameter set to the ratio of malicious input samples (i.e., 15.82%). The \(\gamma\) parameter was optimized through hyperparameter tuning via a simple grid search. Serialized to disk in Joblib format, the best ML model is 486,655 bytes large. The confusion matrix is depicted in Figure 6. With 202,272 false positives, this model performs worse than the previous two models. The classifier has an accuracy of 0.85215, and an f1 score of 0.68154. This time, however, the adversarial DDoS examples with the modified HTTP responses are now correctly classified as malicious. Figure 5: Confusion matrix for the random forest model. #### 3.3.5 Sharing a Multilayer Perceptron Model as Neural Network-Based Threat Intelligence

So far, only traditional ML models were shared within MISP, and some analysts may want to share their experience with deep learning models. Therefore, we built a simple multilayer perceptron (MLP) binary classifier. It has three fully connected feed-forward layers, uses the sigmoid activation function in the last layer, and minimizes the binary cross-entropy loss value with the Adam optimizer. The DL model is trained on the same Zeek log feature vectors as before. The serialized model in HDF5 format has a file size of 35,424 bytes. The confusion matrix is shown in Figure 7.

The classifier has an accuracy of 0.99967, and an f1 score of 0.99896.