Emerhyst-20B EXL2-5bpw

Model creator: Undi95 Original model: Emerhyst-20B

Description

5bpw per weight
This is a test for the exllamav2 model

I converted the model using the convert.py script from the exllamav2 repo: https://github.com/turboderp/exllamav2

Its documentation: https://github.com/turboderp/exllamav2/blob/master/doc/convert.md

I used the WikiText-2-v1 dataset for calibration: https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet

Downloads last month: 10

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.