Hugging Face's logo

HuggingFaceH4
/
mistral-7b-grok

Text Generation
Transformers
Safetensors
mistral
alignment-handbook
Generated from Trainer
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
1
vwxyzjn commited on
Commit
7c96ec4
•
1 Parent(s): cd7f38f

Add HuggingFaceH4/mistral-7b-cai-v20.0.grokai.3.2 checkpoint

Browse files
Files changed (11) hide show
  1. README.md +15 -52
  2. all_results.json +12 -19
  3. config.json +1 -1
  4. eval_results.json +7 -14
  5. model-00001-of-00003.safetensors +1 -1
  6. model-00002-of-00003.safetensors +1 -1
  7. model-00003-of-00003.safetensors +1 -1
  8. special_tokens_map.json +1 -7
  9. train_results.json +6 -6
  10. trainer_state.json +0 -0
  11. training_args.bin +1 -1
README.md CHANGED
@@ -1,33 +1,25 @@
1
  ---
2
  license: apache-2.0
3
- base_model: HuggingFaceH4/mistral-7b-cai
4
  tags:
5
  - alignment-handbook
6
  - generated_from_trainer
7
  datasets:
8
- - HuggingFaceH4/ultrafeedback_binarized_fixed
9
  - HuggingFaceH4/grok-conversation-harmless
 
10
  model-index:
11
- - name: mistral-7b-dpo-v21.0grokai.0.3
12
  results: []
13
  ---
14
 
15
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
  should probably proofread and complete it, then remove this comment. -->
17
 
18
- # mistral-7b-dpo-v21.0grokai.0.3
19
 
20
- This model is a fine-tuned version of [HuggingFaceH4/mistral-7b-cai](https://huggingface.co/HuggingFaceH4/mistral-7b-cai) on the HuggingFaceH4/ultrafeedback_binarized_fixed and the HuggingFaceH4/grok-conversation-harmless datasets.
21
  It achieves the following results on the evaluation set:
22
- - Loss: 0.6270
23
- - Rewards/chosen: -7.6611
24
- - Rewards/rejected: -12.0970
25
- - Rewards/accuracies: 0.6925
26
- - Rewards/margins: 4.4359
27
- - Logps/rejected: -310.5013
28
- - Logps/chosen: -278.5390
29
- - Logits/rejected: -2.1614
30
- - Logits/chosen: -2.1988
31
 
32
  ## Model description
33
 
@@ -46,54 +38,25 @@ More information needed
46
  ### Training hyperparameters
47
 
48
  The following hyperparameters were used during training:
49
- - learning_rate: 5e-07
50
- - train_batch_size: 2
51
  - eval_batch_size: 8
52
  - seed: 42
53
  - distributed_type: multi-GPU
54
  - num_devices: 8
55
- - total_train_batch_size: 16
 
56
  - total_eval_batch_size: 64
57
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
58
- - lr_scheduler_type: linear
59
  - lr_scheduler_warmup_ratio: 0.1
60
- - num_epochs: 3
61
 
62
  ### Training results
63
 
64
- | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
65
- |:-------------:|:-----:|:-----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
66
- | 0.5994 | 0.1 | 400 | 0.5895 | 0.3053 | -0.0377 | 0.5950 | 0.3430 | -189.9080 | -198.8744 | -2.6272 | -2.6485 |
67
- | 0.5024 | 0.19 | 800 | 0.5112 | -0.1278 | -1.0425 | 0.6675 | 0.9147 | -199.9562 | -203.2059 | -2.5093 | -2.5329 |
68
- | 0.5728 | 0.29 | 1200 | 0.5324 | -0.7435 | -1.7880 | 0.6425 | 1.0445 | -207.4112 | -209.3627 | -2.4771 | -2.5058 |
69
- | 0.7378 | 0.39 | 1600 | 0.5213 | -1.6390 | -2.9308 | 0.6650 | 1.2918 | -218.8383 | -218.3174 | -2.4237 | -2.4525 |
70
- | 0.7467 | 0.48 | 2000 | 0.5788 | -2.2099 | -3.4247 | 0.6575 | 1.2148 | -223.7781 | -224.0264 | -2.4106 | -2.4441 |
71
- | 0.4646 | 0.58 | 2400 | 0.5309 | -1.1360 | -2.6997 | 0.6500 | 1.5638 | -216.5279 | -213.2871 | -2.3683 | -2.3994 |
72
- | 0.7454 | 0.67 | 2800 | 0.5290 | -1.9997 | -3.5594 | 0.6700 | 1.5597 | -225.1247 | -221.9242 | -2.4289 | -2.4575 |
73
- | 0.6092 | 0.77 | 3200 | 0.5124 | -1.6120 | -3.1452 | 0.6850 | 1.5332 | -220.9823 | -218.0472 | -2.4755 | -2.5024 |
74
- | 0.674 | 0.87 | 3600 | 0.5134 | -2.9907 | -4.6954 | 0.6750 | 1.7046 | -236.4846 | -231.8350 | -2.2564 | -2.2885 |
75
- | 0.5585 | 0.96 | 4000 | 0.5065 | -2.5232 | -4.1851 | 0.6725 | 1.6619 | -231.3815 | -227.1594 | -2.3968 | -2.4273 |
76
- | 0.0829 | 1.06 | 4400 | 0.5306 | -3.8333 | -6.1055 | 0.6950 | 2.2723 | -250.5862 | -240.2602 | -2.2149 | -2.2565 |
77
- | 0.1383 | 1.16 | 4800 | 0.5432 | -3.8147 | -5.7333 | 0.6675 | 1.9186 | -246.8635 | -240.0743 | -2.3301 | -2.3643 |
78
- | 0.1425 | 1.25 | 5200 | 0.5238 | -4.7732 | -7.0560 | 0.6650 | 2.2827 | -260.0906 | -249.6600 | -2.1705 | -2.2021 |
79
- | 0.1053 | 1.35 | 5600 | 0.5298 | -4.8922 | -7.5361 | 0.6900 | 2.6439 | -264.8917 | -250.8497 | -2.2597 | -2.2978 |
80
- | 0.1301 | 1.44 | 6000 | 0.5190 | -4.0353 | -6.5781 | 0.6850 | 2.5428 | -255.3118 | -242.2802 | -2.1606 | -2.1992 |
81
- | 0.0789 | 1.54 | 6400 | 0.5184 | -4.6125 | -7.3571 | 0.6775 | 2.7446 | -263.1015 | -248.0527 | -2.2220 | -2.2593 |
82
- | 0.1274 | 1.64 | 6800 | 0.5138 | -3.9081 | -6.5224 | 0.6650 | 2.6143 | -254.7549 | -241.0087 | -2.3238 | -2.3653 |
83
- | 0.1095 | 1.73 | 7200 | 0.5153 | -4.1355 | -6.9746 | 0.6750 | 2.8392 | -259.2772 | -243.2823 | -2.2983 | -2.3396 |
84
- | 0.1515 | 1.83 | 7600 | 0.5242 | -4.5052 | -7.4464 | 0.6625 | 2.9412 | -263.9946 | -246.9796 | -2.2513 | -2.2896 |
85
- | 0.1152 | 1.93 | 8000 | 0.5280 | -4.5281 | -7.5632 | 0.6825 | 3.0351 | -265.1628 | -247.2084 | -2.2822 | -2.3185 |
86
- | 0.0385 | 2.02 | 8400 | 0.5478 | -4.9592 | -8.1827 | 0.6800 | 3.2235 | -271.3580 | -251.5196 | -2.2850 | -2.3214 |
87
- | 0.0401 | 2.12 | 8800 | 0.5999 | -6.1863 | -10.0632 | 0.6800 | 3.8769 | -290.1624 | -263.7904 | -2.1925 | -2.2326 |
88
- | 0.0327 | 2.21 | 9200 | 0.6190 | -5.6591 | -9.4406 | 0.6925 | 3.7815 | -283.9365 | -258.5182 | -2.1369 | -2.1748 |
89
- | 0.0425 | 2.31 | 9600 | 0.6298 | -7.3701 | -11.3769 | 0.6925 | 4.0068 | -303.3002 | -275.6286 | -2.1410 | -2.1775 |
90
- | 0.0387 | 2.41 | 10000 | 0.6269 | -7.3259 | -11.5280 | 0.6975 | 4.2020 | -304.8104 | -275.1870 | -2.1791 | -2.2169 |
91
- | 0.043 | 2.5 | 10400 | 0.6376 | -7.2239 | -11.5783 | 0.6925 | 4.3544 | -305.3137 | -274.1667 | -2.2301 | -2.2663 |
92
- | 0.0577 | 2.6 | 10800 | 0.6290 | -7.6726 | -11.9683 | 0.6925 | 4.2956 | -309.2136 | -278.6540 | -2.1968 | -2.2342 |
93
- | 0.019 | 2.7 | 11200 | 0.6260 | -7.2301 | -11.5298 | 0.6825 | 4.2997 | -304.8287 | -274.2284 | -2.1623 | -2.2006 |
94
- | 0.0328 | 2.79 | 11600 | 0.6325 | -7.6096 | -12.0115 | 0.6950 | 4.4019 | -309.6460 | -278.0234 | -2.1388 | -2.1767 |
95
- | 0.036 | 2.89 | 12000 | 0.6312 | -7.8237 | -12.2628 | 0.6900 | 4.4391 | -312.1590 | -280.1643 | -2.1641 | -2.2011 |
96
- | 0.0216 | 2.98 | 12400 | 0.6283 | -7.6679 | -12.0919 | 0.6900 | 4.4240 | -310.4496 | -278.6061 | -2.1613 | -2.1986 |
97
 
98
 
99
  ### Framework versions
 
1
  ---
2
  license: apache-2.0
3
+ base_model: mistralai/Mistral-7B-v0.1
4
  tags:
5
  - alignment-handbook
6
  - generated_from_trainer
7
  datasets:
 
8
  - HuggingFaceH4/grok-conversation-harmless
9
+ - HuggingFaceH4/ultrachat_200k
10
  model-index:
11
+ - name: mistral-7b-ift-v20.0.grokai.3.2
12
  results: []
13
  ---
14
 
15
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
  should probably proofread and complete it, then remove this comment. -->
17
 
18
+ # mistral-7b-ift-v20.0.grokai.3.2
19
 
20
+ This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the HuggingFaceH4/grok-conversation-harmless and the HuggingFaceH4/ultrachat_200k datasets.
21
  It achieves the following results on the evaluation set:
22
+ - Loss: 0.9348
 
 
 
 
 
 
 
 
23
 
24
  ## Model description
25
 
 
38
  ### Training hyperparameters
39
 
40
  The following hyperparameters were used during training:
41
+ - learning_rate: 2e-05
42
+ - train_batch_size: 8
43
  - eval_batch_size: 8
44
  - seed: 42
45
  - distributed_type: multi-GPU
46
  - num_devices: 8
47
+ - gradient_accumulation_steps: 4
48
+ - total_train_batch_size: 256
49
  - total_eval_batch_size: 64
50
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
+ - lr_scheduler_type: cosine
52
  - lr_scheduler_warmup_ratio: 0.1
53
+ - num_epochs: 1
54
 
55
  ### Training results
56
 
57
+ | Training Loss | Epoch | Step | Validation Loss |
58
+ |:-------------:|:-----:|:----:|:---------------:|
59
+ | 0.9326 | 1.0 | 545 | 0.9348 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
 
61
 
62
  ### Framework versions
all_results.json CHANGED
@@ -1,21 +1,14 @@
1
  {
2
- "epoch": 3.0,
3
- "eval_logits/chosen": -2.198760986328125,
4
- "eval_logits/rejected": -2.1614327430725098,
5
- "eval_logps/chosen": -278.5389709472656,
6
- "eval_logps/rejected": -310.5012512207031,
7
- "eval_loss": 0.6269853711128235,
8
- "eval_rewards/accuracies": 0.6924999952316284,
9
- "eval_rewards/chosen": -7.6611433029174805,
10
- "eval_rewards/margins": 4.435902118682861,
11
- "eval_rewards/rejected": -12.0970458984375,
12
- "eval_runtime": 132.6261,
13
- "eval_samples": 3156,
14
- "eval_samples_per_second": 23.796,
15
- "eval_steps_per_second": 0.377,
16
- "train_loss": 0.24049862180692672,
17
- "train_runtime": 20948.1804,
18
- "train_samples": 82424,
19
- "train_samples_per_second": 9.519,
20
- "train_steps_per_second": 0.595
21
  }
 
1
  {
2
+ "epoch": 1.0,
3
+ "eval_loss": 0.934799313545227,
4
+ "eval_runtime": 142.2532,
5
+ "eval_samples": 24266,
6
+ "eval_samples_per_second": 109.024,
7
+ "eval_steps_per_second": 1.708,
8
+ "perplexity": 2.5467023177917714,
9
+ "train_loss": 0.9725383741046311,
10
+ "train_runtime": 5277.8235,
11
+ "train_samples": 211055,
12
+ "train_samples_per_second": 26.46,
13
+ "train_steps_per_second": 0.103
 
 
 
 
 
 
 
14
  }
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "HuggingFaceH4/mistral-7b-cai",
3
  "architectures": [
4
  "MistralForCausalLM"
5
  ],
 
1
  {
2
+ "_name_or_path": "mistralai/Mistral-7B-v0.1",
3
  "architectures": [
4
  "MistralForCausalLM"
5
  ],
eval_results.json CHANGED
@@ -1,16 +1,9 @@
1
  {
2
- "epoch": 3.0,
3
- "eval_logits/chosen": -2.198760986328125,
4
- "eval_logits/rejected": -2.1614327430725098,
5
- "eval_logps/chosen": -278.5389709472656,
6
- "eval_logps/rejected": -310.5012512207031,
7
- "eval_loss": 0.6269853711128235,
8
- "eval_rewards/accuracies": 0.6924999952316284,
9
- "eval_rewards/chosen": -7.6611433029174805,
10
- "eval_rewards/margins": 4.435902118682861,
11
- "eval_rewards/rejected": -12.0970458984375,
12
- "eval_runtime": 132.6261,
13
- "eval_samples": 3156,
14
- "eval_samples_per_second": 23.796,
15
- "eval_steps_per_second": 0.377
16
  }
 
1
  {
2
+ "epoch": 1.0,
3
+ "eval_loss": 0.934799313545227,
4
+ "eval_runtime": 142.2532,
5
+ "eval_samples": 24266,
6
+ "eval_samples_per_second": 109.024,
7
+ "eval_steps_per_second": 1.708,
8
+ "perplexity": 2.5467023177917714
 
 
 
 
 
 
 
9
  }
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:82d5f2b63643649c5c57b449db25080bc6c81c2dea922fcb8aab5190ec15f0fd
3
  size 4943162336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c8e5d6996e054b7c10c1f891947d59e5e462db78fc33252fa58424451d706ea
3
  size 4943162336
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f75e305a1c770759a46794fcf331b8a1e2c64700eba20832e1c462cebef8af57
3
  size 4999819336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed331b509cdee31d8323097ac2b434359b7d8fae383c530753695b221139108c
3
  size 4999819336
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b60e0d9e9e37e59c473aa943a82737a311d71b07ef0409a96de40dfa97bdccc1
3
  size 4540516344
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95a38af2f763dfd084506059d5663c66a2410432723f91502dc3d828cad3d822
3
  size 4540516344
special_tokens_map.json CHANGED
@@ -13,13 +13,7 @@
13
  "rstrip": false,
14
  "single_word": false
15
  },
16
- "pad_token": {
17
- "content": "</s>",
18
- "lstrip": false,
19
- "normalized": false,
20
- "rstrip": false,
21
- "single_word": false
22
- },
23
  "unk_token": {
24
  "content": "<unk>",
25
  "lstrip": false,
 
13
  "rstrip": false,
14
  "single_word": false
15
  },
16
+ "pad_token": "</s>",
 
 
 
 
 
 
17
  "unk_token": {
18
  "content": "<unk>",
19
  "lstrip": false,
train_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
- "epoch": 3.0,
3
- "train_loss": 0.24049862180692672,
4
- "train_runtime": 20948.1804,
5
- "train_samples": 82424,
6
- "train_samples_per_second": 9.519,
7
- "train_steps_per_second": 0.595
8
  }
 
1
  {
2
+ "epoch": 1.0,
3
+ "train_loss": 0.9725383741046311,
4
+ "train_runtime": 5277.8235,
5
+ "train_samples": 211055,
6
+ "train_samples_per_second": 26.46,
7
+ "train_steps_per_second": 0.103
8
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff
 
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8e25a30fcdad636c8cc467560e7ea03470e7cb814424c89d88018e2649b3fe57
3
  size 6264
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97393c2a78f330ed3bfc7e5d7c10eb73f641d9d68463a042f14a4296fc7f29d8
3
  size 6264