GGUF File format for llama-2-chat-13b models from Meta AI.
Quantization:
Currently only 2 quants are available in my repository:
| filename |
quantization |
size |
| ggml-llama-2-13b-chat-q4_k_m.gguf |
Q4_K_M |
7.8GB |
| ggml-llama-2-13b-chat-f16.gguf |
f16 |
26GB |
License subject to Meta's original license agreement.
