| license: other | |
| --- | |
| This is a ggml quantized version of [Replit-v2-CodeInstruct-3B](https://huggingface.co/teknium/Replit-v2-CodeInstruct-3B). Quantized to 4bit -> q4_1. | |
| To run inferene you can use ggml directly or ctranformers (bindings/demo repo to be added): https://github.com/marella/ctransformers |