Commit
•
2ce99e9
1
Parent(s):
1c197f8
Update README.md
Browse files
README.md
CHANGED
|
@@ -4,6 +4,9 @@ license: apache-2.0
|
|
| 4 |
|
| 5 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
| 6 |
|
|
|
|
|
|
|
|
|
|
| 7 |
# Synthia-MoE-v3-Mixtral-8x7B
|
| 8 |
|
| 9 |
This is Synthia-MoE-v3 trained on the official Mistral MoE version (Mixtral-8x7B).
|
|
|
|
| 4 |
|
| 5 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
| 6 |
|
| 7 |
+
# Note:
|
| 8 |
+
Model is most likely over-fitted due to higher learning rate. Will fix this issue in the next release.
|
| 9 |
+
|
| 10 |
# Synthia-MoE-v3-Mixtral-8x7B
|
| 11 |
|
| 12 |
This is Synthia-MoE-v3 trained on the official Mistral MoE version (Mixtral-8x7B).
|