mobiuslabsgmbh
/

Llama-3-8b-instruct_2bitgs64_hqq

Text Generation

Model card Files Files and versions Community

mobicham commited on Jul 29

Commit

25d87d5

•

1 Parent(s): 533fcd6

fix fp16 vram

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -18,7 +18,7 @@ Moreover, this model is fully compatible with <a href="https://github.com/micros
 | Models            | fp16| HQQ+ 2-bit/gs-64|
 |:-------------------:|:--------:|:----------------:|
 | Bitrate (Linear layers)    |   16           |  2.63           |
-| VRAM    |   13.1 (GB)       |  4.3 (GB)          |
 ## Model Decoding Speed
 | Models            | fp16| HQQ+ 2-bit/gs-64|

 | Models            | fp16| HQQ+ 2-bit/gs-64|
 |:-------------------:|:--------:|:----------------:|
 | Bitrate (Linear layers)    |   16           |  2.63           |
+| VRAM    |   15.7 (GB)       |  4.3 (GB)          |
 ## Model Decoding Speed
 | Models            | fp16| HQQ+ 2-bit/gs-64|