zgce
/

Mistral-RP-0.1-7B-3.5bpw-hb6-exl2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Edit model card

Mistral-RP-0.1 7B EXL2-3.5bpw

Model creator: Undi95
Original model: Mistral-RP-0.1-7B

Description

3.5bpw per weight
应 Surdo 要求为小显存做的试验模型

I converted the model using the convert.py script from the exllamav2 repo: https://github.com/turboderp/exllamav2

Its documentation: https://github.com/turboderp/exllamav2/blob/master/doc/convert.md

I used the WikiText-2-v1 dataset for calibration: https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet

Downloads last month: 11

Inference API

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.