Update README.md
Browse files
README.md
CHANGED
@@ -14,7 +14,7 @@ Converted and quantized from EleutherAI's Pythia Deduped checkpoints.
|
|
14 |
|
15 |
Notes:
|
16 |
- Converted with ggerganov/ggml's gpt-neox conversion script, and tested with KoboldCpp.
|
17 |
-
- I can't promise that this will work with other frontends
|
18 |
|
19 |
Versions:
|
20 |
|
|
|
14 |
|
15 |
Notes:
|
16 |
- Converted with ggerganov/ggml's gpt-neox conversion script, and tested with KoboldCpp.
|
17 |
+
- I can't promise that this will work, especially with other frontends. I've had problems when generating words like "Alice" or "Hakurei" / "Gensokyo". Could be related to the ggml implementation of GPT-NeoX having a "hacked" tokenizer [(source)](https://github.com/ggerganov/ggml/tree/master/examples/gpt-neox#notes).
|
18 |
|
19 |
Versions:
|
20 |
|