IHaveNoClueAndIMustPost commited on
Commit
6dd0163
1 Parent(s): 5c1ca1b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -6,4 +6,11 @@ pipeline_tag: text-generation
6
  tags:
7
  - text-generation-inference
8
  ---
9
- This is [Llama2-22b](https://huggingface.co/chargoddard/llama2-22b) in a couple of GGML formats. I have no idea what I'm doing so if something doesn't work that's likely on me, not the models themselves.
 
 
 
 
 
 
 
 
6
  tags:
7
  - text-generation-inference
8
  ---
9
+ This is [Llama2-22b](https://huggingface.co/chargoddard/llama2-22b) in a couple of GGML ormats. I have no idea what I'm doing so if something doesn't work that's likely on me, not the models themselves.
10
+
11
+ Approximate VRAM requirements
12
+ MODEL | SIZE | VRAM
13
+ q5_1 | 16.0GB | 21.5GB
14
+ q4_K_M | 12.8GB | 18.3GB
15
+ q3_K_M | 10.0GB | 16.1GB
16
+ q2_K | 9.0GB | 14.5GB