File size: 701 Bytes
1000ebc bb149da 1000ebc 344c908 bb149da e67b83f bb149da a137779 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
---
license: llama2
library_name: gguf
pipeline_tag: text-generation
---
GGUF importance matrix (imatrix) quants for https://huggingface.co/codellama/CodeLlama-70b-Instruct-hf
The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
The template for this model is very sensitive and must be set very precisely.
All whitespace are intended, and special tokens `<s>` and `<step>` must be encodded properly.
| Layers | Context | Template |
| --- | --- | --- |
| <pre>0</pre> | <pre>4096</pre> | <pre>\<s\> Source: system<br><br> {instructions}\<step\> Source: user<br><br> {prompt}\<step\> Source: assistant<br>Destination: user<br><br> {response}</pre> | |