Hugging Face's logo

brucethemoose
/
Yi-34B-200K-elx2-39bpw

Text Generation
Transformers
PyTorch
English
Yi
custom_code
Model card Files Community
brucethemoose commited on
Commit
8f99dff
·
1 Parent(s): 099085d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -8,7 +8,7 @@ See: https://huggingface.co/01-ai/Yi-34B-200K
8
 
9
  Yi-30B-200K quantized to 3.9bpw, which should allow for ~50K context on 24GB GPUs. Ask if you need another size.
10
 
11
- Quantized with 8K rows on a mix of wikitext and my own RP stories.
12
 
13
  Use with --enable-remote-code in text-gen-ui. Load with Exllamav2_HF, 8-bit cache, and *do not* use the `fast_tokenizer` option. The TFS preset seems to work well with Yi.
14
 
 
8
 
9
  Yi-30B-200K quantized to 3.9bpw, which should allow for ~50K context on 24GB GPUs. Ask if you need another size.
10
 
11
+ Quantized with 8K rows on a mix of wikitext, prompt formatting, and my own RP stories.
12
 
13
  Use with --enable-remote-code in text-gen-ui. Load with Exllamav2_HF, 8-bit cache, and *do not* use the `fast_tokenizer` option. The TFS preset seems to work well with Yi.
14