---
base_model: yhavinga/ul2-large-dutch
library_name: peft
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: ul2-large-dutch-finetuned-oba-book-search
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ul2-large-dutch-finetuned-oba-book-search

This model is a fine-tuned version of [yhavinga/ul2-large-dutch](https://huggingface.co/yhavinga/ul2-large-dutch) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 5.2081
- Top-5-accuracy: 0.0

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Top-5-accuracy |
|:-------------:|:------:|:-----:|:---------------:|:--------------:|
| 8.6533        | 0.0170 | 200   | 5.8825          | 0.0            |
| 8.5044        | 0.0339 | 400   | 5.8463          | 0.0            |
| 8.3692        | 0.0509 | 600   | 5.7566          | 0.0            |
| 8.3791        | 0.0678 | 800   | 5.7155          | 0.0            |
| 8.4643        | 0.0848 | 1000  | 5.6913          | 0.0            |
| 8.3406        | 0.1017 | 1200  | 5.6508          | 0.0            |
| 8.2752        | 0.1187 | 1400  | 5.6089          | 0.0            |
| 8.2495        | 0.1357 | 1600  | 5.5855          | 0.0            |
| 8.1244        | 0.1526 | 1800  | 5.5783          | 0.0            |
| 7.8897        | 0.1696 | 2000  | 5.5554          | 0.0            |
| 8.2289        | 0.1865 | 2200  | 5.5600          | 0.0            |
| 7.9151        | 0.2035 | 2400  | 5.5427          | 0.0            |
| 7.9956        | 0.2205 | 2600  | 5.5188          | 0.0            |
| 8.0472        | 0.2374 | 2800  | 5.4894          | 0.0            |
| 8.0208        | 0.2544 | 3000  | 5.4734          | 0.0            |
| 8.2228        | 0.2713 | 3200  | 5.4615          | 0.0            |
| 8.0756        | 0.2883 | 3400  | 5.4534          | 0.0            |
| 7.8076        | 0.3052 | 3600  | 5.4456          | 0.0            |
| 7.8418        | 0.3222 | 3800  | 5.4461          | 0.0            |
| 7.7266        | 0.3392 | 4000  | 5.4373          | 0.0            |
| 8.0607        | 0.3561 | 4200  | 5.4281          | 0.0            |
| 7.7716        | 0.3731 | 4400  | 5.4081          | 0.0            |
| 7.9324        | 0.3900 | 4600  | 5.3989          | 0.0            |
| 7.9461        | 0.4070 | 4800  | 5.3803          | 0.0            |
| 7.8788        | 0.4239 | 5000  | 5.3734          | 0.0            |
| 7.8748        | 0.4409 | 5200  | 5.3667          | 0.0            |
| 7.8891        | 0.4579 | 5400  | 5.3629          | 0.0            |
| 7.9697        | 0.4748 | 5600  | 5.3624          | 0.0            |
| 7.8402        | 0.4918 | 5800  | 5.3463          | 0.0            |
| 7.8671        | 0.5087 | 6000  | 5.3332          | 0.0            |
| 7.6464        | 0.5257 | 6200  | 5.3190          | 0.0            |
| 7.7773        | 0.5426 | 6400  | 5.3144          | 0.0            |
| 7.723         | 0.5596 | 6600  | 5.3052          | 0.0            |
| 7.8489        | 0.5766 | 6800  | 5.2988          | 0.0            |
| 7.7925        | 0.5935 | 7000  | 5.2946          | 0.0            |
| 7.8374        | 0.6105 | 7200  | 5.2924          | 0.0            |
| 7.4971        | 0.6274 | 7400  | 5.2914          | 0.0            |
| 7.6408        | 0.6444 | 7600  | 5.2859          | 0.0            |
| 7.7993        | 0.6614 | 7800  | 5.2770          | 0.0            |
| 7.5283        | 0.6783 | 8000  | 5.2680          | 0.0            |
| 7.4715        | 0.6953 | 8200  | 5.2637          | 0.0            |
| 7.4699        | 0.7122 | 8400  | 5.2624          | 0.0            |
| 7.6275        | 0.7292 | 8600  | 5.2571          | 0.0            |
| 7.4884        | 0.7461 | 8800  | 5.2509          | 0.0            |
| 7.47          | 0.7631 | 9000  | 5.2448          | 0.0            |
| 7.5765        | 0.7801 | 9200  | 5.2324          | 0.0            |
| 7.6802        | 0.7970 | 9400  | 5.2331          | 0.0            |
| 7.4827        | 0.8140 | 9600  | 5.2346          | 0.0            |
| 7.4127        | 0.8309 | 9800  | 5.2346          | 0.0            |
| 7.5217        | 0.8479 | 10000 | 5.2248          | 0.0            |
| 7.4794        | 0.8648 | 10200 | 5.2201          | 0.0            |
| 7.4414        | 0.8818 | 10400 | 5.2179          | 0.0            |
| 7.5095        | 0.8988 | 10600 | 5.2104          | 0.0            |
| 7.4934        | 0.9157 | 10800 | 5.2084          | 0.0            |
| 7.3595        | 0.9327 | 11000 | 5.2098          | 0.0            |
| 7.6305        | 0.9496 | 11200 | 5.2097          | 0.0            |
| 7.5846        | 0.9666 | 11400 | 5.2084          | 0.0            |
| 7.4536        | 0.9836 | 11600 | 5.2081          | 0.0            |


### Framework versions

- PEFT 0.11.0
- Transformers 4.44.2
- Pytorch 1.13.0+cu116
- Datasets 3.0.0
- Tokenizers 0.19.1