Edit model card

Introduction

This model vistagi/Mixtral-8x7b-v0.1-sft is trained with Ultrachat-200K dataset through supervised finetuning using Mixtral-8x7b-v0.1 as the baseline model. The training is done with bfloat16 precision using LoRA.

Details

Used Librarys

  • torch
  • deepspeed
  • pytorch lightning
  • transformers
  • peft
Downloads last month
414
Safetensors
Model size
46.7B params
Tensor type
BF16
·
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train vistagi/Mixtral-8x7b-v0.1-sft