metadata
library_name: transformers
language:
- en
license: apache-2.0
base_model: answerdotai/ModernBERT-base
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
model-index:
- name: DisambertSingleSense-base
results: []
DisambertSingleSense-base
This model is a fine-tuned version of answerdotai/ModernBERT-base on the semcor dataset. It achieves the following results on the evaluation set:
- Loss: 7.8247
- Precision: 0.7569
- Recall: 0.7432
- F1: 0.7500
- Matthews: 0.7427
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: inverse_sqrt
- lr_scheduler_warmup_steps: 1000
- num_epochs: 10
Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Matthews |
|---|---|---|---|---|---|---|---|
| No log | 0 | 0 | 81.7936 | 0.4396 | 0.3681 | 0.4007 | 0.3673 |
| 0.5564 | 1.0 | 28027 | 0.8047 | 0.7521 | 0.7485 | 0.7503 | 0.7480 |
| 0.4256 | 2.0 | 56054 | 1.0294 | 0.7659 | 0.7590 | 0.7624 | 0.7585 |
| 0.2639 | 3.0 | 84081 | 1.6682 | 0.7656 | 0.7480 | 0.7567 | 0.7475 |
| 0.1907 | 4.0 | 112108 | 3.4982 | 0.7703 | 0.7498 | 0.7599 | 0.7493 |
| 0.0368 | 5.0 | 140135 | 5.1443 | 0.7635 | 0.7458 | 0.7546 | 0.7453 |
| 0.0382 | 6.0 | 168162 | 6.3556 | 0.7674 | 0.7463 | 0.7567 | 0.7458 |
| 0.0172 | 7.0 | 196189 | 8.0398 | 0.7548 | 0.7410 | 0.7479 | 0.7405 |
| 0.0172 | 8.0 | 224216 | 7.1042 | 0.7605 | 0.7467 | 0.7536 | 0.7462 |
| 0.0113 | 9.0 | 252243 | 7.6688 | 0.7624 | 0.7467 | 0.7545 | 0.7462 |
| 0.0064 | 10.0 | 280270 | 7.8247 | 0.7569 | 0.7432 | 0.7500 | 0.7427 |
Framework versions
- Transformers 5.2.0
- Pytorch 2.6.0+cu124
- Datasets 4.5.0
- Tokenizers 0.22.2