metadata

base_model: meta-llama/Meta-Llama-3-8B-Instruct
library_name: peft
license: llama3
tags:
  - trl
  - kto
  - generated_from_trainer
model-index:
  - name: llama3_false_positives_0609_KTO_hp_screening_seeds
    results: []

llama3_false_positives_0609_KTO_hp_screening_seeds

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.5033
Eval/rewards/chosen: 2.8767
Eval/logps/chosen: -164.0585
Eval/rewards/rejected: 2.4882
Eval/logps/rejected: -204.9874
Eval/rewards/margins: 0.3885
Eval/kl: 24.2343

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 1
eval_batch_size: 2
seed: 1234
gradient_accumulation_steps: 8
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 6.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.6591	0.96	12	0.5834	0.2291
0.3244	2.0	25	0.5716	15.3529
0.0459	2.96	37	0.5362	20.4863
0.07	4.0	50	0.5089	23.8717
0.0208	4.96	62	0.4999	24.2550
0.0416	5.76	72	0.5033	24.2343

Framework versions

PEFT 0.11.1
Transformers 4.44.0
Pytorch 2.2.0
Datasets 2.20.0
Tokenizers 0.19.1