metadata
inference: false
license: other
model_type: llama
Mistral 7B v0.2 - AWQ GGUF
These files are in GGUF format.
- Model creator: Mistralai
- Original model: Mistral-7B-Instruct-v0.2
The model was converted by the combination of llama.cpp and quantization method AWQ
How to use models in llama.cpp
./main -m Mistral-7b-v0.1-Q2_K.gguf -n 128 --prompt "Once upon a time"