This is exl2 format model.
Yi-34b-200K-alpaca-rpv3-scipy-6bpw-hb6-exl2
- base model: Yi-34B-200K
- LoRA: Yi-34b-alpaca-cot-lora
- LoRA: limarpv3-yi-llama-34b-lora
- LoRA: Yi-34B-Spicyboros-3.1-LoRA
description
- This is test for exllamav2 model version must after Add Yi support
- 6.0bpw
python convert.py -i Yi-34b-200K-alpaca-rpv3-scipy -c exl2/0000.parquet -o Yi-34b-200K-alpaca-rpv3-scipy-4bpw-hb6-exl2 -hb 6 -l 4096 -b 6 - convert doc
- calibration dataset: WikiText-2-v1
- oobabooga/text-generation-webui must add
--trust-remote-codeinto CMD_FLAGS.txt and use ExLlamav2_HF to load model
- Downloads last month
- 18
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The HF Inference API does not support model that require custom code execution.