4bit variants quantizations of airoboros 70b 1.4.1 (https://huggingface.co/jondurbin/airoboros-l2-70b-gpt4-1.4.1), using exllama2.
You can find 4.25bpw (main branch), 4.5bpw and 4.75bpw in each branch.
Update 21/09/2023
Re-quanted all variants with latest exllamav2 version, which fixed some measurement issues.
- Downloads last month
- 3
This model does not have enough activity to be deployed to Inference API (serverless) yet.
Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.