Latest GGML v2 format for LLaMa-7B

README.md

@@ -0,0 +1,29 @@
+---
+inference: false
+license: other
+---
+# LLaMa 7B GGML
+
+This repo contains GGML format model files for the original LLaMa.
+
+These files are for CPU (+ CUDA) inference using [llama.cpp](https://github.com/ggerganov/llama.cpp).
+
+I've uploaded them mostly for my own convenience, allowing me to easily grab them if and when I need them for future testing and comparisons.
+
+## Provided files 
+
+The following formats are included:
+* float16
+* q4_0 - 4-bit
+* q4_1 - 4-bit
+* q5_0 - 5-bit
+* q5_1 - 5-bit
+* q8_0 - 8-bit
+
+## THESE FILES REQUIRE LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!
+
+llama.cpp recently made a breaking change to its quantisation methods.
+
+I have quantised the GGML files in this repo with the latest version. Therefore you will require llama.cpp compiled on May 12th or later (commit `b9fd7ee` or later) to use them.
+
+I will not be providing GGML formats for the older llama.cpp code. They're already uploaded all over HF if you really need them!

llama-7b.fp16.ggml.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:666a4bb533b303bdaf89e1b6a3b6f93535d868de31d903afdc20983dc526c847
+size 13477814912

llama-7b.ggml.q4_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:417111a40c36bff7ae6c6b3f773ac6efdb1c46584ef1077a1f3404d668e3944f
+size 4212859520

llama-7b.ggml.q4_1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fc3f4925923cafe4681370e863319e8ff8f2d760e6b3f5435b415a407aa8d56
+size 5055128192

llama-7b.ggml.q5_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1226673013a28d61acb94d46eeb15d3623bf0f1472a99ecaf0da8076d680fdf8
+size 4633993856

llama-7b.ggml.q5_1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72040d380ab1067dc08c28d5f16269453bf1d4d7172c24424d4300d8474b42b6
+size 5055128192

llama-7b.ggml.q8_0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3e36532ac11c4a63798ac6ec1471c1dc5a89305c9dec0319dfcb7efc146d001
+size 7581934208