End of training
Browse files- README.md +15 -1
- adapter_model.bin +3 -0
README.md
CHANGED
@@ -2,6 +2,7 @@
|
|
2 |
license: apache-2.0
|
3 |
library_name: peft
|
4 |
tags:
|
|
|
5 |
- generated_from_trainer
|
6 |
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
|
7 |
model-index:
|
@@ -88,7 +89,9 @@ weight_decay: 0.0
|
|
88 |
|
89 |
# special-token-all-linear
|
90 |
|
91 |
-
This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on
|
|
|
|
|
92 |
|
93 |
## Model description
|
94 |
|
@@ -121,6 +124,17 @@ The following hyperparameters were used during training:
|
|
121 |
- lr_scheduler_warmup_steps: 10
|
122 |
- num_epochs: 4
|
123 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
124 |
### Framework versions
|
125 |
|
126 |
- PEFT 0.9.0
|
|
|
2 |
license: apache-2.0
|
3 |
library_name: peft
|
4 |
tags:
|
5 |
+
- axolotl
|
6 |
- generated_from_trainer
|
7 |
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
|
8 |
model-index:
|
|
|
89 |
|
90 |
# special-token-all-linear
|
91 |
|
92 |
+
This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on the None dataset.
|
93 |
+
It achieves the following results on the evaluation set:
|
94 |
+
- Loss: 0.0801
|
95 |
|
96 |
## Model description
|
97 |
|
|
|
124 |
- lr_scheduler_warmup_steps: 10
|
125 |
- num_epochs: 4
|
126 |
|
127 |
+
### Training results
|
128 |
+
|
129 |
+
| Training Loss | Epoch | Step | Validation Loss |
|
130 |
+
|:-------------:|:-----:|:----:|:---------------:|
|
131 |
+
| 2.1829 | 0.01 | 1 | 2.1038 |
|
132 |
+
| 0.091 | 0.8 | 151 | 0.0832 |
|
133 |
+
| 0.0741 | 1.58 | 302 | 0.0801 |
|
134 |
+
| 0.0687 | 2.36 | 453 | 0.0801 |
|
135 |
+
| 0.0654 | 3.14 | 604 | 0.0801 |
|
136 |
+
|
137 |
+
|
138 |
### Framework versions
|
139 |
|
140 |
- PEFT 0.9.0
|
adapter_model.bin
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:c7dbc37d9ba8754f50c194b15eddf83d86fb7118c5b9a827bf71806f2d3eb8af
|
3 |
+
size 1938497058
|