README.md

---
license: other
model-index:
- name: Wizard-Vicuna-13B-Uncensored-SpQR
  results:
  - task:
      type: text-generation-inference
      name: Text Generation
    dataset:
      type: c4
      name: C4
    metrics:
    - type: perplexity
      value: 7.354
  - task:
      type: text-generation-inference
      name: Text Generation
    dataset:
      type: wikitext2
      name: WikiText-2
    metrics:
    - type: perplexity
      value: 5.685
  - task:
      type: text-generation-inference
      name: Text Generation
    dataset:
      type: ptb
      name: PTB
    metrics:
    - type: perplexity
      value: 20.822
pipeline_tag: text-generation
---


# Wizard-Vicuna-13B-Uncensored-SpQR

## Overview
This model is an SpQR 4-bit quantization of the original [Wizard-Vicuna-13B-Uncensored-HF](https://huggingface.co/TheBloke/Wizard-Vicuna-13B-Uncensored-HF)

## Quantization Specifications

- **Quantization**: 4-bit, group size of 16, per-channel with scale and zero-point of 3 bits.
- **Outliers**: Threshold set at 0.2.
- **Permutation Order**: `act_order`.
- **Dampening**: Set at 1e0.
- **Sampling**: 128 samples.
- **Logging**: Via [Weights & Biases](https://wandb.ai/hampterbyte/Wizard-Vicuna-13B-Uncensored-SpQR/runs/95vcnhr8/overview).

## Evaluation Metrics

The following perplexity scores were obtained on various datasets:

| Dataset   | Perplexity |
|:---------:|:----------:|
| c4        | 7.354      |
| wikitext2 | 5.685      |
| ptb       | 20.822     |