Edit model card

t5-base-dutch-finetuned-mt5_base_keyword_extraction_dutch

This model is a fine-tuned version of yhavinga/t5-base-dutch on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6065
  • Rouge1: 0.7675
  • Rouge2: 0.5965
  • Rougel: 0.7531
  • Rougelsum: 0.7534

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5.6e-05
  • train_batch_size: 14
  • eval_batch_size: 14
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 28
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
No log 1.0 94 1.2820 0.4332 0.3007 0.4175 0.4174
1.4828 2.0 188 0.9360 0.6075 0.4541 0.5916 0.5912
1.4828 3.0 282 0.7435 0.6542 0.4825 0.6358 0.6366
0.7633 4.0 376 0.6623 0.6867 0.5071 0.6692 0.6697
0.7633 5.0 470 0.6481 0.7061 0.5254 0.6909 0.6913
0.5935 6.0 564 0.6456 0.7155 0.5367 0.6984 0.6995
0.5935 7.0 658 0.6387 0.7162 0.5388 0.6993 0.7001
0.5101 8.0 752 0.6341 0.7247 0.5495 0.7086 0.7102
0.5101 9.0 846 0.6306 0.7335 0.5527 0.7166 0.7176
0.4449 10.0 940 0.6412 0.7324 0.5559 0.7160 0.7166
0.4449 11.0 1034 0.6439 0.7273 0.5513 0.7126 0.7136
0.4001 12.0 1128 0.6294 0.7415 0.5644 0.7266 0.7268
0.4001 13.0 1222 0.6252 0.7447 0.5658 0.7294 0.7296
0.3589 14.0 1316 0.6257 0.7490 0.5743 0.7341 0.7347
0.3589 15.0 1410 0.6132 0.7474 0.5751 0.7339 0.7346
0.3263 16.0 1504 0.6119 0.7616 0.5858 0.7469 0.7470
0.3263 17.0 1598 0.6088 0.7674 0.5945 0.7527 0.7530
0.2989 18.0 1692 0.6108 0.7655 0.5917 0.7510 0.7514
0.2989 19.0 1786 0.6020 0.7681 0.5961 0.7539 0.7545
0.2846 20.0 1880 0.6065 0.7675 0.5965 0.7531 0.7534

Framework versions

  • Transformers 4.37.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.17.1
  • Tokenizers 0.15.2
Downloads last month
2
Safetensors
Model size
223M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for apatidar0/t5-base-dutch-finetuned-mt5_base_keyword_extraction_dutch

Finetuned
(3)
this model