lucyknada commited on
Commit
31a45f7
1 Parent(s): 44e2612

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -19
README.md CHANGED
@@ -27,25 +27,6 @@ Can I ask a question?<|im_end|>
27
  """
28
  ```
29
 
30
- ## Support
31
-
32
- Upstream support has been merged, so gguf-quants work out of the box now!
33
-
34
- <details><summary>old instructions before PR</summary>
35
-
36
- To run inference on this model, you'll need to use Aphrodite, vLLM or EXL2/tabbyAPI, as llama.cpp hasn't yet merged the required pull request to fix the llama3.1 rope_freqs issue with custom head dimensions.
37
-
38
- However, you can work around this by quantizing the model yourself to create a functional GGUF file. Note that until [this PR](https://github.com/ggerganov/llama.cpp/pull/9141) is merged, the context will be limited to 8k tokens.
39
-
40
- To create a working GGUF file, make the following adjustments:
41
-
42
- 1. Remove the `"rope_scaling": {}` entry from `config.json`
43
- 2. Change `"max_position_embeddings"` to `8192` in `config.json`
44
-
45
- These modifications should allow you to use the model with llama.cpp, albeit with the mentioned context limitation.</strike>
46
-
47
- </details><br>
48
-
49
  ## axolotl config
50
 
51
  <details><summary>See axolotl config</summary>
 
27
  """
28
  ```
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ## axolotl config
31
 
32
  <details><summary>See axolotl config</summary>