RAY2L's picture
Upload folder using huggingface_hub
9834c72 verified
metadata
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
  - alignment-handbook
  - generated_from_trainer
datasets:
  - princeton-nlp/llama3-ultrafeedback
model-index:
  - name: llama-3-8b-instruct-simpo
    results: []

llama-3-8b-instruct-simpo

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the princeton-nlp/llama3-ultrafeedback dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7528
  • Original Losses: 2.0491
  • Weight: 0.3713
  • Abs Diff: 3.1759
  • Rewards/chosen: -45.3959
  • Rewards/rejected: -50.3664
  • Rewards/accuracies: 0.6976
  • Rewards/margins: 4.9705
  • Logps/rejected: -20.1465
  • Logps/chosen: -18.1584
  • Logits/rejected: 1.8309
  • Logits/chosen: 1.7177
  • All Logps 1: -7614.6904
  • All Logps 1 Values: -7614.6909
  • All Logps 2: 414.8609
  • All Logps 2 Values: 414.8609

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-06
  • train_batch_size: 2
  • eval_batch_size: 4
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 8
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 128
  • total_eval_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Original Losses Weight Abs Diff Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen All Logps 1 All Logps 1 Values All Logps 2 All Logps 2 Values
0.7506 0.8549 400 0.7528 2.0491 0.3713 3.1759 -45.3959 -50.3664 0.6976 4.9705 -20.1465 -18.1584 1.8309 1.7177 -7614.6904 -7614.6909 414.8609 414.8609

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.2.2+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1