Update READMD.md Ollama section (#5)
Browse files- Update READMD.md Ollama section (6a2cd9d75c47f197f36bea07e91f82b22dcec8e3)
Co-authored-by: Sarin Suriyakoon <pacozaa@users.noreply.huggingface.co>
README.md
CHANGED
@@ -236,6 +236,20 @@ curl --location 'http://localhost:8000/completion' \
|
|
236 |
}'
|
237 |
```
|
238 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
239 |
### GPU Memory Requirements
|
240 |
| **Number of Parameters** | **FP 16 bits** | **8 bits (Quantized)** | **4 bits (Quantized)** | **Example Graphic Card for 4 bits** |
|
241 |
|------------------|----------------|------------------------|------------------------|---------------------------------------------|
|
|
|
236 |
}'
|
237 |
```
|
238 |
|
239 |
+
### Ollama
|
240 |
+
|
241 |
+
There are two ways to run on ollama
|
242 |
+
|
243 |
+
1. From this repo Modelfile and 4 bit quantized gguf
|
244 |
+
```bash
|
245 |
+
ollama create -f ./Modelfile
|
246 |
+
```
|
247 |
+
|
248 |
+
2. From Ollama CLI
|
249 |
+
```bash
|
250 |
+
ollama run pacozaa/openthaigpt
|
251 |
+
```
|
252 |
+
|
253 |
### GPU Memory Requirements
|
254 |
| **Number of Parameters** | **FP 16 bits** | **8 bits (Quantized)** | **4 bits (Quantized)** | **Example Graphic Card for 4 bits** |
|
255 |
|------------------|----------------|------------------------|------------------------|---------------------------------------------|
|