CNEC_2_0_slavicbert
This model is a fine-tuned version of DeepPavlov/bert-base-bg-cs-pl-ru-cased on the cnec dataset. It achieves the following results on the evaluation set:
- Loss: 0.3354
- Precision: 0.8427
- Recall: 0.8737
- F1: 0.8579
- Accuracy: 0.9553
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30
Training results
Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
---|---|---|---|---|---|---|---|
0.288 | 2.22 | 1000 | 0.2461 | 0.7705 | 0.7926 | 0.7814 | 0.9413 |
0.1551 | 4.44 | 2000 | 0.2270 | 0.8116 | 0.8444 | 0.8277 | 0.9503 |
0.0963 | 6.67 | 3000 | 0.2220 | 0.8181 | 0.8623 | 0.8396 | 0.9533 |
0.0619 | 8.89 | 4000 | 0.2520 | 0.8202 | 0.8598 | 0.8395 | 0.9507 |
0.044 | 11.11 | 5000 | 0.2613 | 0.8332 | 0.8680 | 0.8502 | 0.9535 |
0.0283 | 13.33 | 6000 | 0.2734 | 0.8377 | 0.8673 | 0.8522 | 0.9546 |
0.0227 | 15.56 | 7000 | 0.2908 | 0.8390 | 0.8687 | 0.8536 | 0.9546 |
0.0173 | 17.78 | 8000 | 0.3083 | 0.8393 | 0.8670 | 0.8529 | 0.9528 |
0.013 | 20.0 | 9000 | 0.3238 | 0.8333 | 0.8673 | 0.8500 | 0.9522 |
0.0103 | 22.22 | 10000 | 0.3352 | 0.8325 | 0.8712 | 0.8515 | 0.9539 |
0.0091 | 24.44 | 11000 | 0.3299 | 0.8400 | 0.8655 | 0.8526 | 0.9542 |
0.0073 | 26.67 | 12000 | 0.3376 | 0.8387 | 0.8666 | 0.8524 | 0.9535 |
0.0065 | 28.89 | 13000 | 0.3354 | 0.8427 | 0.8737 | 0.8579 | 0.9553 |
Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0
- Downloads last month
- 0
Finetuned from
Evaluation results
- Precision on cnecvalidation set self-reported0.843
- Recall on cnecvalidation set self-reported0.874
- F1 on cnecvalidation set self-reported0.858
- Accuracy on cnecvalidation set self-reported0.955