MeMo_BERT-SA_01 / README.md
yemen2016's picture
Update README.md
33663e1 verified
metadata
base_model: MiMe-MeMo/MeMo-BERT-03
tags:
  - generated_from_trainer
model-index:
  - name: MeMo_BERT-SA
    results: []
language: da
widget:
  - text: >-
      Men endnu sad hun , bleg og skælvende , ved vinduet og lyttede med rædsel
      i sit blikat en den urolige i de !

MeMo_BERT-SA

This model is a fine-tuned version of MiMe-MeMo/MeMo-BERT-03 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5938
  • F1-score: 0.7727

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss F1-score
No log 1.0 265 0.5938 0.7727
0.591 2.0 530 0.9487 0.7515
0.591 3.0 795 1.1555 0.7375
0.1867 4.0 1060 1.2892 0.7773
0.1867 5.0 1325 1.3791 0.7929
0.0582 6.0 1590 1.5941 0.7810
0.0582 7.0 1855 1.8173 0.7751
0.0166 8.0 2120 1.7725 0.7885
0.0166 9.0 2385 1.7669 0.7939
0.0102 10.0 2650 1.7915 0.7933
0.0102 11.0 2915 1.9139 0.7848
0.0088 12.0 3180 1.9446 0.7816
0.0088 13.0 3445 1.9794 0.7793
0.0124 14.0 3710 1.9904 0.7946
0.0124 15.0 3975 2.0188 0.7831
0.0095 16.0 4240 2.0517 0.7850
0.0001 17.0 4505 2.0427 0.7793
0.0001 18.0 4770 2.0205 0.7902
0.0 19.0 5035 2.0280 0.7847
0.0 20.0 5300 2.0466 0.7793

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.17.0
  • Tokenizers 0.15.1