Instruction tune of Yi-34b-200k with Airoboros-3.1 (fp16)
Overview
This is larryvrh/Yi-34B-200K-Llamafied, with instruction tuning performed with Jon Durbin's jondurbin/airoboros-3.1 dataset. That base model is 01-ai/Yi-34B-200k, but using llama2 model definitions and tokenizer to remove any remote code requirements.
This is a (merged) QLoRA fine-tune (rank 64).
The finetune was performed with 1x RTX 6000 Ada (~80 hours to this checkpoint). Prompts were truncated to 4096 tokens (for speed and VRAM headroom).
I have done very little testing with this model, so feedback on real world performance is appreciated!
How to Use
Use as you would any other Hugging Face fp16 llama-2 model.
Prompting:
Model was trained with llama-2 chat prompt format. See jondurbin/airoboros-l2-13b-3.1.1 model card for details.
- Downloads last month
- 928
This model does not have enough activity to be deployed to Inference API (serverless) yet.
Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.