helena-balabin/tweety-ft-nlp-xxl

robbert-2023-dutch-large-ft-nlp-xxl

This model is a fine-tuned version of Tweeties/tweety-7b-dutch-v24a on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.6377

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2000

Training results

Training Loss	Epoch	Step	Validation Loss
4.5391	0.0424	50	4.0393
4.0619	0.0849	100	3.9164
3.9954	0.1273	150	3.8674
3.9471	0.1698	200	3.8336
3.9038	0.2122	250	3.8172
3.9226	0.2547	300	3.7907
3.8362	0.2971	350	3.7700
3.8002	0.3396	400	3.7757
3.8245	0.3820	450	3.7469
3.7602	0.4244	500	3.7271
3.7623	0.4669	550	3.7159
3.7735	0.5093	600	3.7080
3.7444	0.5518	650	3.7013
3.7844	0.5942	700	3.6899
3.7439	0.6367	750	3.6784
3.6924	0.6791	800	3.6739
3.7393	0.7216	850	3.6651
3.7169	0.7640	900	3.6671
3.718	0.8065	950	3.6556
3.7342	0.8489	1000	3.6514
3.7327	0.8913	1050	3.6444
3.7048	0.9338	1100	3.6417
3.7166	0.9762	1150	3.6396
3.5797	1.0187	1200	3.6392
3.4572	1.0611	1250	3.6405
3.4068	1.1036	1300	3.6497
3.4344	1.1460	1350	3.6510
3.4347	1.1885	1400	3.6452
3.3956	1.2309	1450	3.6482
3.4001	1.2733	1500	3.6559
3.394	1.3158	1550	3.6457
3.421	1.3582	1600	3.6468
3.4062	1.4007	1650	3.6457
3.3814	1.4431	1700	3.6435
3.3415	1.4856	1750	3.6458
3.3699	1.5280	1800	3.6402
3.3902	1.5705	1850	3.6390
3.3718	1.6129	1900	3.6382
3.4026	1.6553	1950	3.6380
3.3431	1.6978	2000	3.6377

Framework versions

• PEFT 0.11.1
• Transformers 4.41.2
• Pytorch 2.1.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1