|
--- |
|
license: mit |
|
--- |
|
|
|
# Phi-3 MoE mini 4k instruct raw |
|
|
|
The is a 8x MoE version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct). |
|
It is based on the Llamafied version [vonjack/Phi-3-mini-4k-instruct-LLaMAfied](https://huggingface.co/vonjack/Phi-3-mini-4k-instruct-LLaMAfied) of |
|
[Gan Feng](https://huggingface.co/vonjack). |
|
|
|
It was created with the help of [mergekit](https://github.com/arcee-ai/mergekit) with this |
|
[configuration](https://huggingface.co/PhilipMay/Phi-3-MoE-mini-4k-instruct-raw/blob/main/mergekit_moe_config.yml) and this command: |
|
|
|
```bash |
|
TODO |
|
``` |
|
|
|
As the router was initialized randomly during merging, this is a raw model. |
|
It should be trained before it can be used. |
|
|
|
## Licensing |
|
|
|
Copyright (c) 2024 [Philip May](https://philipmay.org)\ |
|
Copyright (c) [Gan Feng](https://huggingface.co/vonjack)\ |
|
Copyright (c) Microsoft Corporation |
|
|
|
Licensed under the **MIT License** (the "License"); you may not use this file except in compliance with the License. |
|
You may obtain a copy of the License by reviewing the file |
|
[LICENSE](https://huggingface.co/PhilipMay/Phi-3-MoE-mini-4k-instruct-raw/blob/main/LICENSE) in the repository. |
|
|
