Hugging Face's logo

YeungNLP
/
firefly-qwen1.5-en-7b-unsloth

Text Generation
Transformers
Safetensors
qwen2
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
YeungNLP commited on
Commit
1e4e472
•
1 Parent(s): bb212b4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -22
README.md CHANGED
@@ -158,25 +158,24 @@ The following hyperparameters were used during DPO:
158
 
159
  The table below shows the full set of DPO training metrics:
160
 
161
- | Epoch | Step | Loss | Rewards/accuracies | Rewards/margins | Rewards/chosen | Rewards/rejected | Logits/chosen| Logits/rejected | Logps/chosen| Logps/rejected|
162
- |---|---|---|---|---|---|---|---|---|---|---|
163
- |0.05|100|0.6128|0.6572|0.3914|-0.0622|-0.4537|1.107|1.1104|-283.7632|-264.5925|
164
- |0.1|200|0.6066|0.6913|0.662|-0.3589|-1.0209|0.9433|0.9431|-279.0002|-268.6432|
165
- |0.16|300|0.5803|0.7069|0.876|-0.3849|-1.2609|0.8411|0.8537|-289.9482|-274.3425|
166
- |0.21|400|0.5624|0.7169|0.9575|-0.2447|-1.2022|0.7615|0.7497|-293.8072|-274.4167|
167
- |0.26|500|0.5863|0.7|0.8908|-0.5283|-1.4191|0.537|0.5085|-284.3388|-267.9294|
168
- |0.31|600|0.5612|0.7166|1.0791|-0.592|-1.6711|0.7121|0.7219|-293.2425|-278.5992|
169
-
170
- |0.37|700|0.5741|0.7234|1.0742|-0.8469|-1.9211|0.6002|0.5769|-300.8099|-285.9137|
171
- |0.42|800|0.582|0.7141|1.0414|-1.1658|-2.2072|0.7191|0.5934|-300.458|-286.1|
172
- |0.47|900|0.5694|0.7178|1.2055|-1.7372|-2.9426|0.4226|0.316|-305.5303|-290.7548|
173
- |0.52|1000|0.5827|0.7134|1.1063|-1.354|-2.4603|0.535|0.4022|-302.7598|-286.636|
174
- |0.58|1100|0.5553|0.7306|1.3631|-1.5861|-2.9492|0.7636|0.6559|-312.9375|-290.3474|
175
- |0.63|1200|0.5633|0.7341|1.2689|-1.7187|-2.9876|0.6555|0.5894|-315.0179|-298.2406|
176
- |0.68|1300|0.5705|0.7284|1.3501|-1.7762|-3.1263|0.7419|0.6874|-310.9056|-294.2934|
177
- |0.73|1400|0.5458|0.7347|1.4555|-2.2377|-3.6932|0.7279|0.6564|-309.141|-299.1613|
178
- |0.79|1500|0.5797|0.7222|1.2937|-2.4483|-3.742|0.8444|0.771|-321.578|-298.111|
179
- |0.84|1600|0.5572|0.7319|1.4824|-2.9344|-4.4168|0.9202|0.8605|-323.4034|-307.0114|
180
- |0.89|1700|0.5518|0.7281|1.4263|-2.7301|-4.1564|0.9257|0.8785|-313.694|-298.1267|
181
- |0.94|1800|0.5572|0.7272|1.5121|-2.9505|-4.4627|0.7899|0.7503|-314.1552|-305.9873|
182
- |0.99|1900|0.5763|0.7241|1.4982|-2.7064|-4.2047|0.7841|0.7023|-310.6677|-299.5064|
 
158
 
159
  The table below shows the full set of DPO training metrics:
160
 
161
+ | Epoch | Step | Loss | Rewards/accuracies | Rewards/margins | Rewards/chosen | Rewards/rejected | Logits/chosen | Logits/rejected | Logps/chosen | Logps/rejected |
162
+ |-------|------|--------|--------------------|-----------------|----------------|------------------|---------------|-----------------|--------------|----------------|
163
+ | 0.05 | 100 | 0.6128 | 0.6572 | 0.3914 | -0.0622 | -0.4537 | 1.107 | 1.1104 | -283.7632 | -264.5925 |
164
+ | 0.1 | 200 | 0.6066 | 0.6913 | 0.662 | -0.3589 | -1.0209 | 0.9433 | 0.9431 | -279.0002 | -268.6432 |
165
+ | 0.16 | 300 | 0.5803 | 0.7069 | 0.876 | -0.3849 | -1.2609 | 0.8411 | 0.8537 | -289.9482 | -274.3425 |
166
+ | 0.21 | 400 | 0.5624 | 0.7169 | 0.9575 | -0.2447 | -1.2022 | 0.7615 | 0.7497 | -293.8072 | -274.4167 |
167
+ | 0.26 | 500 | 0.5863 | 0.7 | 0.8908 | -0.5283 | -1.4191 | 0.537 | 0.5085 | -284.3388 | -267.9294 |
168
+ | 0.31 | 600 | 0.5612 | 0.7166 | 1.0791 | -0.592 | -1.6711 | 0.7121 | 0.7219 | -293.2425 | -278.5992 |
169
+ | 0.37 | 700 | 0.5741 | 0.7234 | 1.0742 | -0.8469 | -1.9211 | 0.6002 | 0.5769 | -300.8099 | -285.9137 |
170
+ | 0.42 | 800 | 0.582 | 0.7141 | 1.0414 | -1.1658 | -2.2072 | 0.7191 | 0.5934 | -300.458 | -286.1 |
171
+ | 0.47 | 900 | 0.5694 | 0.7178 | 1.2055 | -1.7372 | -2.9426 | 0.4226 | 0.316 | -305.5303 | -290.7548 |
172
+ | 0.52 | 1000 | 0.5827 | 0.7134 | 1.1063 | -1.354 | -2.4603 | 0.535 | 0.4022 | -302.7598 | -286.636 |
173
+ | 0.58 | 1100 | 0.5553 | 0.7306 | 1.3631 | -1.5861 | -2.9492 | 0.7636 | 0.6559 | -312.9375 | -290.3474 |
174
+ | 0.63 | 1200 | 0.5633 | 0.7341 | 1.2689 | -1.7187 | -2.9876 | 0.6555 | 0.5894 | -315.0179 | -298.2406 |
175
+ | 0.68 | 1300 | 0.5705 | 0.7284 | 1.3501 | -1.7762 | -3.1263 | 0.7419 | 0.6874 | -310.9056 | -294.2934 |
176
+ | 0.73 | 1400 | 0.5458 | 0.7347 | 1.4555 | -2.2377 | -3.6932 | 0.7279 | 0.6564 | -309.141 | -299.1613 |
177
+ | 0.79 | 1500 | 0.5797 | 0.7222 | 1.2937 | -2.4483 | -3.742 | 0.8444 | 0.771 | -321.578 | -298.111 |
178
+ | 0.84 | 1600 | 0.5572 | 0.7319 | 1.4824 | -2.9344 | -4.4168 | 0.9202 | 0.8605 | -323.4034 | -307.0114 |
179
+ | 0.89 | 1700 | 0.5518 | 0.7281 | 1.4263 | -2.7301 | -4.1564 | 0.9257 | 0.8785 | -313.694 | -298.1267 |
180
+ | 0.94 | 1800 | 0.5572 | 0.7272 | 1.5121 | -2.9505 | -4.4627 | 0.7899 | 0.7503 | -314.1552 | -305.9873 |
181
+ | 0.99 | 1900 | 0.5763 | 0.7241 | 1.4982 | -2.7064 | -4.2047 | 0.7841 | 0.7023 | -310.6677 | -299.5064 |