Phi-3 MoE mini 4k instruct raw
The is a 8x MoE version of microsoft/Phi-3-mini-4k-instruct. It is based on the Llamafied version vonjack/Phi-3-mini-4k-instruct-LLaMAfied of Gan Feng.
It was created with the help of mergekit with this configuration and this command:
TODO
As the router was initialized randomly during merging, this is a raw model. It should be trained before it can be used.
Licensing
Copyright (c) 2024 Philip May
Copyright (c) Gan Feng
Copyright (c) Microsoft Corporation
Licensed under the MIT License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License by reviewing the file LICENSE in the repository.
- Downloads last month
- 3