File size: 1,479 Bytes
4607d69 33f2fce 08fbe13 bad5b0b 19343c3 08fbe13 19343c3 08fbe13 bad5b0b 19343c3 08fbe13 d884be2 4607d69 33f2fce f24b28a 33f2fce 893e6f4 33f2fce 893e6f4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
---
license: other
model-index:
- name: Wizard-Vicuna-13B-Uncensored-SpQR
results:
- task:
type: text-generation-inference
name: Text Generation
dataset:
type: c4
name: C4
metrics:
- type: perplexity
value: 7.354
- task:
type: text-generation-inference
name: Text Generation
dataset:
type: wikitext2
name: WikiText-2
metrics:
- type: perplexity
value: 5.685
- task:
type: text-generation-inference
name: Text Generation
dataset:
type: ptb
name: PTB
metrics:
- type: perplexity
value: 20.822
pipeline_tag: text-generation
---
# Wizard-Vicuna-13B-Uncensored-SpQR
## Overview
This model is an SpQR 4-bit quantization of the original [Wizard-Vicuna-13B-Uncensored-HF](https://huggingface.co/TheBloke/Wizard-Vicuna-13B-Uncensored-HF)
## Quantization Specifications
- **Quantization**: 4-bit, group size of 16, per-channel with scale and zero-point of 3 bits.
- **Outliers**: Threshold set at 0.2.
- **Permutation Order**: `act_order`.
- **Dampening**: Set at 1e0.
- **Sampling**: 128 samples.
- **Logging**: Via [Weights & Biases](https://wandb.ai/hampterbyte/Wizard-Vicuna-13B-Uncensored-SpQR/runs/95vcnhr8/overview).
## Evaluation Metrics
The following perplexity scores were obtained on various datasets:
| Dataset | Perplexity |
|:---------:|:----------:|
| c4 | 7.354 |
| wikitext2 | 5.685 |
| ptb | 20.822 | |