Text Generation
Transformers
Safetensors
English
llama
conversational
Inference Endpoints
text-generation-inference
Edit model card

quant of ehartford's dolphin-2.2-yi-34b-200k

done with exllamav2 0.0.11 so should have lower loss than earlier quants

python3 convert.py \
    -i /input/ehartford_dolphin-2.2-yi-34b-200k/ \
    -c /input/pippa_cleaned/0000.parquet \
    -o /output/temp/ \
    -cf /output/dolphin-2.2-yi-34b-200k-4bpw-h6-exl2/ \
    -l 8192 \
    -ml 8192 \
    -b 4 \
    -hb 6
Downloads last month
1

Datasets used to train waldie/dolphin-2.2-yi-34b-200k-4bpw-h6-exl2