File size: 311 Bytes
1d47997 bae228c c672fd3 |
1 2 3 4 5 6 7 8 |
---
license: other
---
---
This is a ggml quantized version of [Replit-v2-CodeInstruct-3B](https://huggingface.co/teknium/Replit-v2-CodeInstruct-3B). Quantized to 4bit -> q4_1.
To run inferene you can use ggml directly or ctranformers (bindings/demo repo to be added): https://github.com/marella/ctransformers |