Training Data
#1
by
dennisc1
- opened
Hi Jorge,
I've been waiting for a Dutch mixtral finetune, very cool! Could you say how you trained the model? Which data did you use?
It's been trained on:
https://huggingface.co/datasets/Rijgersberg/no_robots_nl
https://huggingface.co/datasets/Rijgersberg/ultrachat_10k_nl
https://huggingface.co/datasets/BramVanroy/dutch_chat_datasets
It trained for about 3 epochs on 1 H100 but after evaluation the hellaswag_nl it dropped in performance.
So use with caution!
I will try at a later moment after I read the Mixtral paper