Hugging Face's logo

cynthiachan
/
finetuned-bert-base-10pct

Token Classification
Transformers
PyTorch
bert
Generated from Trainer
Inference Endpoints
Model card Files Community
1
finetuned-bert-base-10pct / README.md
cynthiachan's picture
cynthiachan
update model
2d6d9e1
preview code
|
raw
history blame contribute delete
11.1 kB
metadata
license: apache-2.0
tags:
  - generated_from_trainer
datasets:
  - cynthiachan/FeedRef_10pct
model-index:
  - name: training
    results: []

training

This model is a fine-tuned version of bert-base-cased on the cynthiachan/FeedRef_10pct dataset. It achieves the following results on the evaluation set:

  • Loss: 0.1291
  • Attackid Precision: 1.0
  • Attackid Recall: 1.0
  • Attackid F1: 1.0
  • Attackid Number: 6
  • Cve Precision: 0.8333
  • Cve Recall: 0.9091
  • Cve F1: 0.8696
  • Cve Number: 11
  • Defenderthreat Precision: 0.0
  • Defenderthreat Recall: 0.0
  • Defenderthreat F1: 0.0
  • Defenderthreat Number: 2
  • Domain Precision: 0.7826
  • Domain Recall: 0.7826
  • Domain F1: 0.7826
  • Domain Number: 23
  • Email Precision: 0.6667
  • Email Recall: 0.6667
  • Email F1: 0.6667
  • Email Number: 3
  • Filepath Precision: 0.6766
  • Filepath Recall: 0.8242
  • Filepath F1: 0.7432
  • Filepath Number: 165
  • Hostname Precision: 1.0
  • Hostname Recall: 0.9167
  • Hostname F1: 0.9565
  • Hostname Number: 12
  • Ipv4 Precision: 0.8333
  • Ipv4 Recall: 0.8333
  • Ipv4 F1: 0.8333
  • Ipv4 Number: 12
  • Md5 Precision: 0.7246
  • Md5 Recall: 0.9615
  • Md5 F1: 0.8264
  • Md5 Number: 52
  • Sha1 Precision: 0.0667
  • Sha1 Recall: 0.1429
  • Sha1 F1: 0.0909
  • Sha1 Number: 7
  • Sha256 Precision: 0.6780
  • Sha256 Recall: 0.9091
  • Sha256 F1: 0.7767
  • Sha256 Number: 44
  • Uri Precision: 0.0
  • Uri Recall: 0.0
  • Uri F1: 0.0
  • Uri Number: 1
  • Overall Precision: 0.6910
  • Overall Recall: 0.8402
  • Overall F1: 0.7583
  • Overall Accuracy: 0.9725

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3.0

Training results

Training Loss Epoch Step Validation Loss Attackid Precision Attackid Recall Attackid F1 Attackid Number Cve Precision Cve Recall Cve F1 Cve Number Defenderthreat Precision Defenderthreat Recall Defenderthreat F1 Defenderthreat Number Domain Precision Domain Recall Domain F1 Domain Number Email Precision Email Recall Email F1 Email Number Filepath Precision Filepath Recall Filepath F1 Filepath Number Hostname Precision Hostname Recall Hostname F1 Hostname Number Ipv4 Precision Ipv4 Recall Ipv4 F1 Ipv4 Number Md5 Precision Md5 Recall Md5 F1 Md5 Number Sha1 Precision Sha1 Recall Sha1 F1 Sha1 Number Sha256 Precision Sha256 Recall Sha256 F1 Sha256 Number Uri Precision Uri Recall Uri F1 Uri Number Overall Precision Overall Recall Overall F1 Overall Accuracy
0.3943 0.37 500 0.2881 0.0 0.0 0.0 6 0.0 0.0 0.0 11 0.0 0.0 0.0 2 0.0 0.0 0.0 23 0.0 0.0 0.0 3 0.1138 0.2 0.1451 165 0.0692 0.9167 0.1287 12 0.4706 0.6667 0.5517 12 0.75 0.9231 0.8276 52 0.0 0.0 0.0 7 0.5694 0.9318 0.7069 44 0.0 0.0 0.0 1 0.2342 0.4172 0.3 0.9360
0.1987 0.75 1000 0.1722 0.5 1.0 0.6667 6 1.0 1.0 1.0 11 0.0 0.0 0.0 2 0.0 0.0 0.0 23 0.0 0.0 0.0 3 0.4779 0.6545 0.5524 165 0.25 0.6667 0.3636 12 0.6923 0.75 0.7200 12 0.6364 0.9423 0.7597 52 0.0 0.0 0.0 7 0.6545 0.8182 0.7273 44 0.0 0.0 0.0 1 0.5136 0.6716 0.5821 0.9529
0.1595 1.12 1500 0.1346 0.8571 1.0 0.9231 6 1.0 1.0 1.0 11 0.0 0.0 0.0 2 0.4286 0.5217 0.4706 23 0.0 0.0 0.0 3 0.5797 0.7273 0.6452 165 0.44 0.9167 0.5946 12 0.3929 0.9167 0.55 12 0.6364 0.9423 0.7597 52 0.0 0.0 0.0 7 0.78 0.8864 0.8298 44 0.0 0.0 0.0 1 0.5768 0.7663 0.6582 0.9658
0.118 1.5 2000 0.1436 1.0 1.0 1.0 6 1.0 1.0 1.0 11 0.0 0.0 0.0 2 0.6087 0.6087 0.6087 23 0.0 0.0 0.0 3 0.6101 0.8061 0.6945 165 0.9091 0.8333 0.8696 12 0.7273 0.6667 0.6957 12 0.7869 0.9231 0.8496 52 0.2143 0.4286 0.2857 7 0.7407 0.9091 0.8163 44 0.0 0.0 0.0 1 0.6675 0.8077 0.7309 0.9686
0.1198 1.87 2500 0.1385 1.0 1.0 1.0 6 0.7692 0.9091 0.8333 11 0.0 0.0 0.0 2 0.85 0.7391 0.7907 23 0.0 0.0 0.0 3 0.6390 0.7939 0.7081 165 1.0 0.8333 0.9091 12 0.5333 0.6667 0.5926 12 0.7778 0.9423 0.8522 52 0.3333 0.5714 0.4211 7 0.8571 0.9545 0.9032 44 0.0 0.0 0.0 1 0.6995 0.8195 0.7548 0.9687
0.0742 2.25 3000 0.1291 1.0 1.0 1.0 6 0.8333 0.9091 0.8696 11 0.0 0.0 0.0 2 0.7826 0.7826 0.7826 23 0.6667 0.6667 0.6667 3 0.6766 0.8242 0.7432 165 1.0 0.9167 0.9565 12 0.8333 0.8333 0.8333 12 0.7246 0.9615 0.8264 52 0.0667 0.1429 0.0909 7 0.6780 0.9091 0.7767 44 0.0 0.0 0.0 1 0.6910 0.8402 0.7583 0.9725
0.0687 2.62 3500 0.1385 1.0 1.0 1.0 6 1.0 1.0 1.0 11 0.0 0.0 0.0 2 0.8077 0.9130 0.8571 23 1.0 1.0 1.0 3 0.7746 0.8121 0.7929 165 0.7333 0.9167 0.8148 12 0.7143 0.8333 0.7692 12 0.96 0.9231 0.9412 52 0.4444 0.5714 0.5 7 0.8113 0.9773 0.8866 44 0.0 0.0 0.0 1 0.8083 0.8609 0.8338 0.9737
0.0652 3.0 4000 0.1299 1.0 1.0 1.0 6 1.0 1.0 1.0 11 0.0 0.0 0.0 2 0.8077 0.9130 0.8571 23 1.0 1.0 1.0 3 0.7553 0.8606 0.8045 165 0.8462 0.9167 0.8800 12 0.7143 0.8333 0.7692 12 0.8571 0.9231 0.8889 52 0.75 0.8571 0.8000 7 0.8723 0.9318 0.9011 44 0.0 0.0 0.0 1 0.8038 0.8846 0.8423 0.9772

Framework versions

  • Transformers 4.21.2
  • Pytorch 1.12.1+cu102
  • Datasets 2.4.0
  • Tokenizers 0.12.1