cybersecurity-ner / README.md
bnsapa's picture
Add new example (#3)
1ad52f1
metadata
license: apache-2.0
base_model: distilbert-base-uncased
tags:
  - generated_from_trainer
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: cybersecurity-ner
    results: []
datasets:
  - bnsapa/cybersecurity-ner
language:
  - en
library_name: transformers
widget:
  - text: microsoft and google are working to build AI models
  - text: >-
      Having obtained the necessary permissions from the user, Riltok contacts
      its C&C server.
  - text: Tweets in Twitter can be controversial
  - text: >-
      xyz is custom virus gains access to the messages in the victim's mobile
      and contacts the attacker's server

cybersecurity-ner

This model is a fine-tuned version of distilbert-base-uncased on the cybersecurity-ner dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2196
  • Precision: 0.7942
  • Recall: 0.7925
  • F1: 0.7933
  • Accuracy: 0.9508

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 8

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
No log 1.0 167 0.2492 0.6870 0.7406 0.7128 0.9293
No log 2.0 334 0.2026 0.7733 0.7346 0.7534 0.9420
0.2118 3.0 501 0.1895 0.7735 0.7934 0.7833 0.9493
0.2118 4.0 668 0.1834 0.7785 0.8189 0.7982 0.9511
0.2118 5.0 835 0.2060 0.8113 0.7965 0.8039 0.9522
0.0507 6.0 1002 0.2153 0.7692 0.8226 0.7950 0.9511
0.0507 7.0 1169 0.2141 0.7866 0.7962 0.7914 0.9507
0.0507 8.0 1336 0.2196 0.7942 0.7925 0.7933 0.9508

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu118
  • Datasets 2.15.0
  • Tokenizers 0.15.0