File size: 865 Bytes
57ffeb1 caf25ad 57ffeb1 caf25ad 57ffeb1 caf25ad 57ffeb1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
tags:
- LLaMA
- GGML
---
# LLAMA-GGML-v2
This is repo for LLaMA models quantised down to 4bit for the latest [llama.cpp](https://github.com/ggerganov/llama.cpp) GGML v2 format.
## THE FILES REQUIRES LATEST LLAMA.CPP (May 12th 2023 - commit b9fd7ee)!
llama.cpp recently made a breaking change to its quantisation methods.
I have quantised the GGML files in this repo with the latest version.
## How to run in `text-generation-webui`
GGML models can be loaded into text-generation-webui by installing the llama.cpp module, then placing the ggml model file in a model folder as usual.
Further instructions here: [text-generation-webui/docs/llama.cpp-models.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md).
Note: at this time text-generation-webui may not support the new May 12th llama.cpp quantisation methods. |