qwp4w3hyb commited on
Commit
8c2dea2
1 Parent(s): 27c2641

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -23,7 +23,8 @@ It seems llama 3 uses a different eos tokens depending if it is in instruct mode
23
  The initial upload had some issues with this as it uses the "default" eos token of 128001, but when in instruct mode llama only outputs 128009 as eos token which causes it to ramble on and on without stopping.
24
 
25
  I have uploaded fixed quants with the eos token id manually set to 128009.
26
- This fixes the issue for me, but you have to make sure to use the correct chat template, I recommend using [this](https://github.com/ggerganov/llama.cpp/pull/6751) PR and then launching llama.cpp with `--chat-template llama3`.
 
27
 
28
  If you do not want to redownload you can fix your local gguf file with this command:
29
  ```
 
23
  The initial upload had some issues with this as it uses the "default" eos token of 128001, but when in instruct mode llama only outputs 128009 as eos token which causes it to ramble on and on without stopping.
24
 
25
  I have uploaded fixed quants with the eos token id manually set to 128009.
26
+
27
+ This fixes the issue for me, but you have to make sure to use the correct chat template, ~I recommend using [this](https://github.com/ggerganov/llama.cpp/pull/6751) PR~(it has been merged just used the newest llama.cpp master) and then launching llama.cpp with `--chat-template llama3`.
28
 
29
  If you do not want to redownload you can fix your local gguf file with this command:
30
  ```