IHaveNoClueAndIMustPost
commited on
Commit
•
6dd0163
1
Parent(s):
5c1ca1b
Update README.md
Browse files
README.md
CHANGED
@@ -6,4 +6,11 @@ pipeline_tag: text-generation
|
|
6 |
tags:
|
7 |
- text-generation-inference
|
8 |
---
|
9 |
-
This is [Llama2-22b](https://huggingface.co/chargoddard/llama2-22b) in a couple of GGML
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
tags:
|
7 |
- text-generation-inference
|
8 |
---
|
9 |
+
This is [Llama2-22b](https://huggingface.co/chargoddard/llama2-22b) in a couple of GGML ormats. I have no idea what I'm doing so if something doesn't work that's likely on me, not the models themselves.
|
10 |
+
|
11 |
+
Approximate VRAM requirements
|
12 |
+
MODEL | SIZE | VRAM
|
13 |
+
q5_1 | 16.0GB | 21.5GB
|
14 |
+
q4_K_M | 12.8GB | 18.3GB
|
15 |
+
q3_K_M | 10.0GB | 16.1GB
|
16 |
+
q2_K | 9.0GB | 14.5GB
|