Hugging Face's logo

TomGrc
/
FusionNet_7Bx2_MoE_v0.1

Text Generation
Transformers
Safetensors
English
mixtral
Mixture of Experts
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
1
FusionNet_7Bx2_MoE_v0.1 / README.md
TomGrc's picture
TomGrc
Update README.md
644451f verified
preview code
|
raw
history blame
450 Bytes
metadata
license: mit
language:
  - en
pipeline_tag: text-generation
tags:
  - moe

FusionNet_7Bx2_MoE_v0.1

Fine-tuned model on English language using MoE method. The improved version from FusionNet_7Bx2_MoE_14B.

Model description

The FusionNet_7Bx2_MoE_v0.1 is a model to experiment with the MoE method, which could significantly increase the performance of the original model. The FusionNet has 12.9B parameters, and this model is fine-tuned. Enjoy!