kaitchup
/

Yi-6B-gptq-4bit

Text Generation

Inference Endpoints

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Edit model card

Model Details

This is 01-ai/Yi-6B quantized and serialized with AutoGPTQ in 4-bit.

Details here:

Yi: Fine-tune and Run One the Best Bilingual LLMs on Your Computer

Developed by: The Kaitchup
Language(s) (NLP): English
License: yi-license https://huggingface.co/01-ai/Yi-6B/blob/main/LICENSE

Downloads last month: 5

Safetensors

Model size

1.27B params

Tensor type

I32

·

FP16

·

Collection including kaitchup/Yi-6B-gptq-4bit

Yi

Quantized and fine-tuned versions of the Yi models • 4 items • Updated Mar 17