---
tags:
- quantized
- 2-bit
- 3-bit
- 4-bit
- 5-bit
- 6-bit
- 8-bit
- fp16
- GGUF
- transformers
- pytorch
- yi
- text-generation
- conversational
- endpoints_compatible
- text-generation-inference
- text-generation
license: apache-2.0
library_name: transformers
inference: false
pipeline_tag: text-generation
---
The gguf quantization of [Fi-9B](https://huggingface.co/wenbopan/Fi-9B-200K)