---
license: mit
---

# Phi-3 MoE mini 4k instruct raw

The is a 8x MoE version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct).
It is based on the Llamafied version [vonjack/Phi-3-mini-4k-instruct-LLaMAfied](https://huggingface.co/vonjack/Phi-3-mini-4k-instruct-LLaMAfied) of 
[Gan Feng](https://huggingface.co/vonjack).

It was created with the help of [mergekit](https://github.com/arcee-ai/mergekit) with this
[configuration](https://huggingface.co/PhilipMay/Phi-3-MoE-mini-4k-instruct-raw/blob/main/mergekit_moe_config.yml) and this command:

```bash
TODO
```

As the router was initialized randomly during merging, this is a raw model.
It should be trained before it can be used.

## Licensing

Copyright (c) 2024 [Philip May](https://philipmay.org)\
Copyright (c) [Gan Feng](https://huggingface.co/vonjack)\
Copyright (c) Microsoft Corporation

Licensed under the **MIT License** (the "License"); you may not use this file except in compliance with the License.
You may obtain a copy of the License by reviewing the file
[LICENSE](https://huggingface.co/PhilipMay/Phi-3-MoE-mini-4k-instruct-raw/blob/main/LICENSE) in the repository.