metadata
license: other
This is a ggml quantized version of Replit-v2-CodeInstruct-3B. Quantized to 4bit -> q4_1. To run inferene you can use ggml directly or ctranformers (bindings/demo repo to be added): https://github.com/marella/ctransformers