Hugging Face's logo

waldie
/
Euryale-1.3-L2-70B-2.18bpw-h6-exl2

Text Generation
Transformers
Safetensors
English
llama
text-generation-inference
Inference Endpoints
Model card Files Community
1
Edit model card

quant of Sao10K's Euryale-1.3-L2-70B

fits into 24gb at 8k context lol

python3 convert.py \
    -i /input/Sao10K_Euryale-1.3-L2-70B/ \
    -c /input/wikitext/0000.parquet \
    -o /output/temp/ \
    -cf /output/2.18bpw/ \
    -b 2.18 \
    -hb 6
Downloads last month
64
Inference API
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.