Hugging Face's logo

altomek
/
RE-70B-AS3D

Text Generation
Transformers
Safetensors
English
llama
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
altomek commited on
Commit
d27cace
·
verified ·
1 Parent(s): e73fea9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -39,7 +39,11 @@ Use Llama 3 template, model with ALpaca template sometimes halucinates and gener
39
 
40
  ### Quants
41
 
42
- More quants are comming soon... and I need to redo them most likely as they do not encode chat template in tokenizer_config...
 
 
 
 
 
43
 
44
- - [GGUF](https://huggingface.co/altomek/RE-70B-AS3D-GGUF)
45
 
 
39
 
40
  ### Quants
41
 
42
+ More quants are comming soon... and I need to redo GGUF quants(!) as they do not encode chat template in tokenizer_config... if your inference engine uses chat template from GGUF file you will se halucinations. However they work fine for me in SillyTawern...
43
+
44
+ - [GGUF](https://huggingface.co/altomek/RE-70B-AS3D-GGUF) --> TO BE UPDATED!
45
+ - [3.5 BPW](https://huggingface.co/altomek/RE-70B-AS3D-3.5bpw-EXL2)
46
+ - [3.75 BPW](https://huggingface.co/altomek/RE-70B-AS3D-3.75bpw-EXL2)
47
+ - [4 BPW](https://huggingface.co/altomek/RE-70B-AS3D-4bpw-EXL2)
48
 
 
49