YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
Original model: https://huggingface.co/Doctor-Shotgun/Nous-Capybara-limarpv3-34B
Using erotiquant-xl.parquet as calibration dataset.
5bpw - Can run 10k context in ~23.3GB VRAM with 8bit-cache option.
Prompt format - extended alpaca with length modifier:
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Input:
{input}
### Response: (length = long)
- Downloads last month
- 14
This model does not have enough activity to be deployed to Inference API (serverless) yet.
Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.