README.md

---
inference: false
license: other
---
# LLaMa 7B GGML

This repo contains GGML format model files for the original LLaMa.

These files are for CPU (+ CUDA) inference using [llama.cpp](https://github.com/ggerganov/llama.cpp).

I've uploaded them mostly for my own convenience, allowing me to easily grab them if and when I need them for future testing and comparisons.

## Provided files 

The following formats are included:
* float16
* q4_0 - 4-bit
* q4_1 - 4-bit
* q5_0 - 5-bit
* q5_1 - 5-bit
* q8_0 - 8-bit

## THESE FILES REQUIRE LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!

llama.cpp recently made a breaking change to its quantisation methods.

I have quantised the GGML files in this repo with the latest version. Therefore you will require llama.cpp compiled on May 12th or later (commit `b9fd7ee` or later) to use them.

I will not be providing GGML formats for the older llama.cpp code. They're already uploaded all over HF if you really need them!