--- license: apache-2.0 tags: - generated_from_trainer datasets: - cynthiachan/FeedRef_10pct model-index: - name: training results: [] --- # training This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the cynthiachan/FeedRef_10pct dataset. It achieves the following results on the evaluation set: - Loss: 0.1291 - Attackid Precision: 1.0 - Attackid Recall: 1.0 - Attackid F1: 1.0 - Attackid Number: 6 - Cve Precision: 0.8333 - Cve Recall: 0.9091 - Cve F1: 0.8696 - Cve Number: 11 - Defenderthreat Precision: 0.0 - Defenderthreat Recall: 0.0 - Defenderthreat F1: 0.0 - Defenderthreat Number: 2 - Domain Precision: 0.7826 - Domain Recall: 0.7826 - Domain F1: 0.7826 - Domain Number: 23 - Email Precision: 0.6667 - Email Recall: 0.6667 - Email F1: 0.6667 - Email Number: 3 - Filepath Precision: 0.6766 - Filepath Recall: 0.8242 - Filepath F1: 0.7432 - Filepath Number: 165 - Hostname Precision: 1.0 - Hostname Recall: 0.9167 - Hostname F1: 0.9565 - Hostname Number: 12 - Ipv4 Precision: 0.8333 - Ipv4 Recall: 0.8333 - Ipv4 F1: 0.8333 - Ipv4 Number: 12 - Md5 Precision: 0.7246 - Md5 Recall: 0.9615 - Md5 F1: 0.8264 - Md5 Number: 52 - Sha1 Precision: 0.0667 - Sha1 Recall: 0.1429 - Sha1 F1: 0.0909 - Sha1 Number: 7 - Sha256 Precision: 0.6780 - Sha256 Recall: 0.9091 - Sha256 F1: 0.7767 - Sha256 Number: 44 - Uri Precision: 0.0 - Uri Recall: 0.0 - Uri F1: 0.0 - Uri Number: 1 - Overall Precision: 0.6910 - Overall Recall: 0.8402 - Overall F1: 0.7583 - Overall Accuracy: 0.9725 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 3.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Attackid Precision | Attackid Recall | Attackid F1 | Attackid Number | Cve Precision | Cve Recall | Cve F1 | Cve Number | Defenderthreat Precision | Defenderthreat Recall | Defenderthreat F1 | Defenderthreat Number | Domain Precision | Domain Recall | Domain F1 | Domain Number | Email Precision | Email Recall | Email F1 | Email Number | Filepath Precision | Filepath Recall | Filepath F1 | Filepath Number | Hostname Precision | Hostname Recall | Hostname F1 | Hostname Number | Ipv4 Precision | Ipv4 Recall | Ipv4 F1 | Ipv4 Number | Md5 Precision | Md5 Recall | Md5 F1 | Md5 Number | Sha1 Precision | Sha1 Recall | Sha1 F1 | Sha1 Number | Sha256 Precision | Sha256 Recall | Sha256 F1 | Sha256 Number | Uri Precision | Uri Recall | Uri F1 | Uri Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |:-------------:|:-----:|:----:|:---------------:|:------------------:|:---------------:|:-----------:|:---------------:|:-------------:|:----------:|:------:|:----------:|:------------------------:|:---------------------:|:-----------------:|:---------------------:|:----------------:|:-------------:|:---------:|:-------------:|:---------------:|:------------:|:--------:|:------------:|:------------------:|:---------------:|:-----------:|:---------------:|:------------------:|:---------------:|:-----------:|:---------------:|:--------------:|:-----------:|:-------:|:-----------:|:-------------:|:----------:|:------:|:----------:|:--------------:|:-----------:|:-------:|:-----------:|:----------------:|:-------------:|:---------:|:-------------:|:-------------:|:----------:|:------:|:----------:|:-----------------:|:--------------:|:----------:|:----------------:| | 0.3943 | 0.37 | 500 | 0.2881 | 0.0 | 0.0 | 0.0 | 6 | 0.0 | 0.0 | 0.0 | 11 | 0.0 | 0.0 | 0.0 | 2 | 0.0 | 0.0 | 0.0 | 23 | 0.0 | 0.0 | 0.0 | 3 | 0.1138 | 0.2 | 0.1451 | 165 | 0.0692 | 0.9167 | 0.1287 | 12 | 0.4706 | 0.6667 | 0.5517 | 12 | 0.75 | 0.9231 | 0.8276 | 52 | 0.0 | 0.0 | 0.0 | 7 | 0.5694 | 0.9318 | 0.7069 | 44 | 0.0 | 0.0 | 0.0 | 1 | 0.2342 | 0.4172 | 0.3 | 0.9360 | | 0.1987 | 0.75 | 1000 | 0.1722 | 0.5 | 1.0 | 0.6667 | 6 | 1.0 | 1.0 | 1.0 | 11 | 0.0 | 0.0 | 0.0 | 2 | 0.0 | 0.0 | 0.0 | 23 | 0.0 | 0.0 | 0.0 | 3 | 0.4779 | 0.6545 | 0.5524 | 165 | 0.25 | 0.6667 | 0.3636 | 12 | 0.6923 | 0.75 | 0.7200 | 12 | 0.6364 | 0.9423 | 0.7597 | 52 | 0.0 | 0.0 | 0.0 | 7 | 0.6545 | 0.8182 | 0.7273 | 44 | 0.0 | 0.0 | 0.0 | 1 | 0.5136 | 0.6716 | 0.5821 | 0.9529 | | 0.1595 | 1.12 | 1500 | 0.1346 | 0.8571 | 1.0 | 0.9231 | 6 | 1.0 | 1.0 | 1.0 | 11 | 0.0 | 0.0 | 0.0 | 2 | 0.4286 | 0.5217 | 0.4706 | 23 | 0.0 | 0.0 | 0.0 | 3 | 0.5797 | 0.7273 | 0.6452 | 165 | 0.44 | 0.9167 | 0.5946 | 12 | 0.3929 | 0.9167 | 0.55 | 12 | 0.6364 | 0.9423 | 0.7597 | 52 | 0.0 | 0.0 | 0.0 | 7 | 0.78 | 0.8864 | 0.8298 | 44 | 0.0 | 0.0 | 0.0 | 1 | 0.5768 | 0.7663 | 0.6582 | 0.9658 | | 0.118 | 1.5 | 2000 | 0.1436 | 1.0 | 1.0 | 1.0 | 6 | 1.0 | 1.0 | 1.0 | 11 | 0.0 | 0.0 | 0.0 | 2 | 0.6087 | 0.6087 | 0.6087 | 23 | 0.0 | 0.0 | 0.0 | 3 | 0.6101 | 0.8061 | 0.6945 | 165 | 0.9091 | 0.8333 | 0.8696 | 12 | 0.7273 | 0.6667 | 0.6957 | 12 | 0.7869 | 0.9231 | 0.8496 | 52 | 0.2143 | 0.4286 | 0.2857 | 7 | 0.7407 | 0.9091 | 0.8163 | 44 | 0.0 | 0.0 | 0.0 | 1 | 0.6675 | 0.8077 | 0.7309 | 0.9686 | | 0.1198 | 1.87 | 2500 | 0.1385 | 1.0 | 1.0 | 1.0 | 6 | 0.7692 | 0.9091 | 0.8333 | 11 | 0.0 | 0.0 | 0.0 | 2 | 0.85 | 0.7391 | 0.7907 | 23 | 0.0 | 0.0 | 0.0 | 3 | 0.6390 | 0.7939 | 0.7081 | 165 | 1.0 | 0.8333 | 0.9091 | 12 | 0.5333 | 0.6667 | 0.5926 | 12 | 0.7778 | 0.9423 | 0.8522 | 52 | 0.3333 | 0.5714 | 0.4211 | 7 | 0.8571 | 0.9545 | 0.9032 | 44 | 0.0 | 0.0 | 0.0 | 1 | 0.6995 | 0.8195 | 0.7548 | 0.9687 | | 0.0742 | 2.25 | 3000 | 0.1291 | 1.0 | 1.0 | 1.0 | 6 | 0.8333 | 0.9091 | 0.8696 | 11 | 0.0 | 0.0 | 0.0 | 2 | 0.7826 | 0.7826 | 0.7826 | 23 | 0.6667 | 0.6667 | 0.6667 | 3 | 0.6766 | 0.8242 | 0.7432 | 165 | 1.0 | 0.9167 | 0.9565 | 12 | 0.8333 | 0.8333 | 0.8333 | 12 | 0.7246 | 0.9615 | 0.8264 | 52 | 0.0667 | 0.1429 | 0.0909 | 7 | 0.6780 | 0.9091 | 0.7767 | 44 | 0.0 | 0.0 | 0.0 | 1 | 0.6910 | 0.8402 | 0.7583 | 0.9725 | | 0.0687 | 2.62 | 3500 | 0.1385 | 1.0 | 1.0 | 1.0 | 6 | 1.0 | 1.0 | 1.0 | 11 | 0.0 | 0.0 | 0.0 | 2 | 0.8077 | 0.9130 | 0.8571 | 23 | 1.0 | 1.0 | 1.0 | 3 | 0.7746 | 0.8121 | 0.7929 | 165 | 0.7333 | 0.9167 | 0.8148 | 12 | 0.7143 | 0.8333 | 0.7692 | 12 | 0.96 | 0.9231 | 0.9412 | 52 | 0.4444 | 0.5714 | 0.5 | 7 | 0.8113 | 0.9773 | 0.8866 | 44 | 0.0 | 0.0 | 0.0 | 1 | 0.8083 | 0.8609 | 0.8338 | 0.9737 | | 0.0652 | 3.0 | 4000 | 0.1299 | 1.0 | 1.0 | 1.0 | 6 | 1.0 | 1.0 | 1.0 | 11 | 0.0 | 0.0 | 0.0 | 2 | 0.8077 | 0.9130 | 0.8571 | 23 | 1.0 | 1.0 | 1.0 | 3 | 0.7553 | 0.8606 | 0.8045 | 165 | 0.8462 | 0.9167 | 0.8800 | 12 | 0.7143 | 0.8333 | 0.7692 | 12 | 0.8571 | 0.9231 | 0.8889 | 52 | 0.75 | 0.8571 | 0.8000 | 7 | 0.8723 | 0.9318 | 0.9011 | 44 | 0.0 | 0.0 | 0.0 | 1 | 0.8038 | 0.8846 | 0.8423 | 0.9772 | ### Framework versions - Transformers 4.21.2 - Pytorch 1.12.1+cu102 - Datasets 2.4.0 - Tokenizers 0.12.1