results

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0651

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2
• training_steps: 75

Training results

Training Loss	Epoch	Step	Validation Loss
1.9743	0.8163	5	1.9198
1.8581	1.6327	10	1.7523
1.5766	2.4490	15	1.5925
1.4205	3.2653	20	1.4522
1.3769	4.0816	25	1.3597
1.1971	4.8980	30	1.2998
1.1704	5.7143	35	1.2477
1.128	6.5306	40	1.2061
1.0483	7.3469	45	1.1649
1.017	8.1633	50	1.1308
0.8937	8.9796	55	1.1093
0.8445	9.7959	60	1.0892
0.8322	10.6122	65	1.0763
0.926	11.4286	70	1.0652
0.7692	12.2449	75	1.0651

Framework versions

• PEFT 0.11.2.dev0
• Transformers 4.43.0.dev0
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1