File size: 1,187 Bytes
775190c 2451cf8 2ba03d7 2db2c49 39e924b 2451cf8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
---
license: mit
---
# Phi-3 MoE mini 4k instruct raw
The is a 8x MoE version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct).
It is based on the Llamafied version [vonjack/Phi-3-mini-4k-instruct-LLaMAfied](https://huggingface.co/vonjack/Phi-3-mini-4k-instruct-LLaMAfied) of
[Gan Feng](https://huggingface.co/vonjack).
It was created with the help of [mergekit](https://github.com/arcee-ai/mergekit) with this
[configuration](https://huggingface.co/PhilipMay/Phi-3-MoE-mini-4k-instruct-raw/blob/main/mergekit_moe_config.yml) and this command:
```bash
TODO
```
As the router was initialized randomly during merging, this is a raw model.
It should be trained before it can be used.
## Licensing
Copyright (c) 2024 [Philip May](https://philipmay.org)\
Copyright (c) [Gan Feng](https://huggingface.co/vonjack)\
Copyright (c) Microsoft Corporation
Licensed under the **MIT License** (the "License"); you may not use this file except in compliance with the License.
You may obtain a copy of the License by reviewing the file
[LICENSE](https://huggingface.co/PhilipMay/Phi-3-MoE-mini-4k-instruct-raw/blob/main/LICENSE) in the repository.
|