Edit model card

Meow. This an experimental mixture of expert model with just 2 experts based on Llama 3 Instruct plain in combo with finetune. Specifically, it is built on top of the Meta-Llama-3-8B-Instruct model and finetune is trained on Argilla Capybara dataset.

Experimental mixture of 2 experts Llama3-8b-Instruct

Built with Llama 3

Downloads last month
11
Safetensors
Model size
13.7B params
Tensor type
BF16
·
Inference API
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train nisten/llama3-2x8b-MoE-41k-experiment1