Edit model card

tulu-2-7b-full-UF-5e-7

This model is a fine-tuned version of allenai/tulu-2-7b on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9017
  • Rewards/chosen: -4.8659
  • Rewards/rejected: -5.8048
  • Rewards/accuracies: 0.6230
  • Rewards/margins: 0.9389
  • Rewards/margins Max: 5.6516
  • Rewards/margins Min: -2.8163
  • Rewards/margins Std: 2.7854
  • Logps/rejected: -916.6636
  • Logps/chosen: -832.4283
  • Logits/rejected: 0.4957
  • Logits/chosen: 0.2899

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-07
  • train_batch_size: 2
  • eval_batch_size: 8
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 8
  • total_train_batch_size: 16
  • total_eval_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Rewards/margins Max Rewards/margins Min Rewards/margins Std Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.6816 0.07 100 0.6919 0.0000 -0.0020 0.5417 0.0021 0.0277 -0.0245 0.0175 -336.3843 -345.8331 -1.1956 -1.1695
0.5468 0.15 200 0.6793 -0.1136 -0.1432 0.5794 0.0296 0.2495 -0.1965 0.1511 -350.5013 -357.1989 -1.1509 -1.1466
0.3597 0.22 300 0.6788 -0.9347 -1.0641 0.5714 0.1294 1.0084 -0.7320 0.5779 -442.5906 -439.3020 -1.0512 -1.0629
0.2059 0.29 400 0.7172 -1.9680 -2.3061 0.5972 0.3381 2.3443 -1.3886 1.2205 -566.7862 -542.6320 -0.8695 -0.8807
0.1354 0.37 500 0.8082 -3.1553 -3.7843 0.6190 0.6290 4.0818 -2.2017 2.0321 -714.6080 -661.3674 -0.1617 -0.2554
0.1327 0.44 600 0.8436 -3.8517 -4.6192 0.6190 0.7675 4.8313 -2.4317 2.3526 -798.1056 -731.0093 0.1600 0.0173
0.0777 0.52 700 0.9893 -4.9432 -5.9282 0.6190 0.9850 6.3532 -3.2959 3.1250 -929.0052 -840.1605 0.6301 0.4163
0.0638 0.59 800 0.8086 -3.8655 -4.6357 0.6190 0.7702 4.5021 -2.2919 2.2427 -799.7516 -732.3853 0.2889 0.1244
0.0997 0.66 900 0.8639 -4.4406 -5.3058 0.6270 0.8652 5.1592 -2.6378 2.5658 -866.7603 -789.8954 0.3918 0.2055
0.0708 0.74 1000 0.8618 -4.4546 -5.2895 0.6230 0.8349 5.0604 -2.6224 2.5213 -865.1302 -791.2946 0.4063 0.2199
0.141 0.81 1100 0.9049 -4.8648 -5.7977 0.6190 0.9330 5.6327 -2.8439 2.7856 -915.9548 -832.3105 0.5083 0.3017
0.0775 0.88 1200 0.9049 -4.9040 -5.8585 0.6210 0.9546 5.7130 -2.8316 2.8132 -922.0319 -836.2313 0.5172 0.3074
0.0464 0.96 1300 0.9017 -4.8659 -5.8048 0.6230 0.9389 5.6516 -2.8163 2.7854 -916.6636 -832.4283 0.4957 0.2899

Framework versions

  • Transformers 4.39.0.dev0
  • Pytorch 2.1.0+cu121
  • Datasets 2.14.6
  • Tokenizers 0.15.2
Downloads last month
1
Safetensors
Model size
6.74B params
Tensor type
BF16
·

Finetuned from