Llama-2-7b-chat-hf-finetune_90_10_SY_gold

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1470

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 3
• eval_batch_size: 3
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: reduce_lr_on_plateau
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.5446	0.9937	157	0.7173
0.0863	1.9873	314	0.8235
0.1229	2.9810	471	0.8690
0.0914	3.9747	628	0.9089
0.0766	4.9684	785	0.9324
0.0842	5.9620	942	0.9564
0.0725	6.9557	1099	0.9903
0.0584	7.9494	1256	1.0348
0.0653	8.9430	1413	1.0468
0.0601	9.9367	1570	1.0397
0.0601	10.9304	1727	1.0622
0.0595	11.9241	1884	1.0757
0.0676	12.9177	2041	1.0911
0.0513	13.9114	2198	1.0942
0.0492	14.9051	2355	1.0980
0.0489	15.8987	2512	1.1041
0.05	16.8924	2669	1.1095
0.0498	17.8861	2826	1.1136
0.0494	18.8797	2983	1.1174
0.0483	19.8734	3140	1.1220
0.0476	20.8671	3297	1.1226
0.0487	21.8608	3454	1.1250
0.048	22.8544	3611	1.1297
0.0445	23.8481	3768	1.1307
0.0461	24.8418	3925	1.1321
0.0456	25.8354	4082	1.1338
0.0455	26.8291	4239	1.1355
0.0437	27.8228	4396	1.1372
0.0434	28.8165	4553	1.1390
0.0442	29.8101	4710	1.1400
0.0432	30.8038	4867	1.1417
0.0433	31.7975	5024	1.1426
0.0424	32.7911	5181	1.1437
0.0415	33.7848	5338	1.1453
0.0424	34.7785	5495	1.1453
0.0399	35.7722	5652	1.1455
0.04	36.7658	5809	1.1457
0.0401	37.7595	5966	1.1458
0.0394	38.7532	6123	1.1461
0.0369	39.7468	6280	1.1462
0.0515	40.7405	6437	1.1464
0.0511	41.7342	6594	1.1465
0.0483	42.7278	6751	1.1465
0.0478	43.7215	6908	1.1468
0.0492	44.7152	7065	1.1469
0.0492	45.7089	7222	1.1469
0.0473	46.7025	7379	1.1469
0.0483	47.6962	7536	1.1470
0.0474	48.6899	7693	1.1470
0.0473	49.6835	7850	1.1470

Framework versions

• PEFT 0.11.1
• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1