esahit's picture
Rerun first training run on increased dataset
6569b85 verified
|
raw
history blame
5.45 kB
metadata
base_model: yhavinga/ul2-large-dutch
library_name: peft
license: apache-2.0
tags:
  - generated_from_trainer
model-index:
  - name: ul2-large-dutch-finetuned-oba-book-search
    results: []

ul2-large-dutch-finetuned-oba-book-search

This model is a fine-tuned version of yhavinga/ul2-large-dutch on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 5.2081
  • Top-5-accuracy: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.001
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Top-5-accuracy
8.6533 0.0170 200 5.8825 0.0
8.5044 0.0339 400 5.8463 0.0
8.3692 0.0509 600 5.7566 0.0
8.3791 0.0678 800 5.7155 0.0
8.4643 0.0848 1000 5.6913 0.0
8.3406 0.1017 1200 5.6508 0.0
8.2752 0.1187 1400 5.6089 0.0
8.2495 0.1357 1600 5.5855 0.0
8.1244 0.1526 1800 5.5783 0.0
7.8897 0.1696 2000 5.5554 0.0
8.2289 0.1865 2200 5.5600 0.0
7.9151 0.2035 2400 5.5427 0.0
7.9956 0.2205 2600 5.5188 0.0
8.0472 0.2374 2800 5.4894 0.0
8.0208 0.2544 3000 5.4734 0.0
8.2228 0.2713 3200 5.4615 0.0
8.0756 0.2883 3400 5.4534 0.0
7.8076 0.3052 3600 5.4456 0.0
7.8418 0.3222 3800 5.4461 0.0
7.7266 0.3392 4000 5.4373 0.0
8.0607 0.3561 4200 5.4281 0.0
7.7716 0.3731 4400 5.4081 0.0
7.9324 0.3900 4600 5.3989 0.0
7.9461 0.4070 4800 5.3803 0.0
7.8788 0.4239 5000 5.3734 0.0
7.8748 0.4409 5200 5.3667 0.0
7.8891 0.4579 5400 5.3629 0.0
7.9697 0.4748 5600 5.3624 0.0
7.8402 0.4918 5800 5.3463 0.0
7.8671 0.5087 6000 5.3332 0.0
7.6464 0.5257 6200 5.3190 0.0
7.7773 0.5426 6400 5.3144 0.0
7.723 0.5596 6600 5.3052 0.0
7.8489 0.5766 6800 5.2988 0.0
7.7925 0.5935 7000 5.2946 0.0
7.8374 0.6105 7200 5.2924 0.0
7.4971 0.6274 7400 5.2914 0.0
7.6408 0.6444 7600 5.2859 0.0
7.7993 0.6614 7800 5.2770 0.0
7.5283 0.6783 8000 5.2680 0.0
7.4715 0.6953 8200 5.2637 0.0
7.4699 0.7122 8400 5.2624 0.0
7.6275 0.7292 8600 5.2571 0.0
7.4884 0.7461 8800 5.2509 0.0
7.47 0.7631 9000 5.2448 0.0
7.5765 0.7801 9200 5.2324 0.0
7.6802 0.7970 9400 5.2331 0.0
7.4827 0.8140 9600 5.2346 0.0
7.4127 0.8309 9800 5.2346 0.0
7.5217 0.8479 10000 5.2248 0.0
7.4794 0.8648 10200 5.2201 0.0
7.4414 0.8818 10400 5.2179 0.0
7.5095 0.8988 10600 5.2104 0.0
7.4934 0.9157 10800 5.2084 0.0
7.3595 0.9327 11000 5.2098 0.0
7.6305 0.9496 11200 5.2097 0.0
7.5846 0.9666 11400 5.2084 0.0
7.4536 0.9836 11600 5.2081 0.0

Framework versions

  • PEFT 0.11.0
  • Transformers 4.44.2
  • Pytorch 1.13.0+cu116
  • Datasets 3.0.0
  • Tokenizers 0.19.1