Update README.md
Browse files
README.md
CHANGED
@@ -5,4 +5,6 @@ license: other
|
|
5 |
---
|
6 |
|
7 |
This is a [ggml](https://github.com/ggerganov/ggml/) quantized version of [Replit-v2-CodeInstruct-3B](https://huggingface.co/teknium/Replit-v2-CodeInstruct-3B). Quantized to 4bit -> q4_1.
|
8 |
-
To run inference you can use ggml directly or ctransformers (bindings/demo repo to be added): https://github.com/marella/ctransformers
|
|
|
|
|
|
5 |
---
|
6 |
|
7 |
This is a [ggml](https://github.com/ggerganov/ggml/) quantized version of [Replit-v2-CodeInstruct-3B](https://huggingface.co/teknium/Replit-v2-CodeInstruct-3B). Quantized to 4bit -> q4_1.
|
8 |
+
To run inference you can use ggml directly or ctransformers (bindings/demo repo to be added): https://github.com/marella/ctransformers.
|
9 |
+
|
10 |
+
Memory usage of model: **2GB~**
|