Honkware's picture
Update README.md
893e6f4
metadata
license: other
model-index:
  - name: Wizard-Vicuna-13B-Uncensored-SpQR
    results:
      - task:
          type: text-generation-inference
          name: Text Generation
        dataset:
          type: c4
          name: C4
        metrics:
          - type: perplexity
            value: 7.354
      - task:
          type: text-generation-inference
          name: Text Generation
        dataset:
          type: wikitext2
          name: WikiText-2
        metrics:
          - type: perplexity
            value: 5.685
      - task:
          type: text-generation-inference
          name: Text Generation
        dataset:
          type: ptb
          name: PTB
        metrics:
          - type: perplexity
            value: 20.822
pipeline_tag: text-generation

Wizard-Vicuna-13B-Uncensored-SpQR

Overview

This model is an SpQR 4-bit quantization of the original Wizard-Vicuna-13B-Uncensored-HF

Quantization Specifications

  • Quantization: 4-bit, group size of 16, per-channel with scale and zero-point of 3 bits.
  • Outliers: Threshold set at 0.2.
  • Permutation Order: act_order.
  • Dampening: Set at 1e0.
  • Sampling: 128 samples.
  • Logging: Via Weights & Biases.

Evaluation Metrics

The following perplexity scores were obtained on various datasets:

Dataset Perplexity
c4 7.354
wikitext2 5.685
ptb 20.822