metadata
license: gpl-3.0
metrics:
- perplexity
pipeline_tag: conversational
tags:
- LLaMa
- text-generation-inference
- ggml
LLaMa 65B converted to ggml via LLaMa.cpp, then quantized to 4bit.
I recommend the following settings when running as a good starting point: main.exe -m ggml-LLaMa-65B-q4_0.bin -n -1 -t 42 -c 2048 --temp 0.35 --interactive-first --repeat_penalty 1.2 --instruct --color