Honkware commited on
Commit
33f2fce
1 Parent(s): 4607d69

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -0
README.md CHANGED
@@ -1,3 +1,59 @@
1
  ---
2
  license: other
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: other
3
+ homepage: https://wandb.ai/hampterbyte/Wizard-Vicuna-13B-Uncensored-SpQR/runs/95vcnhr8/overview
4
+ model-index:
5
+ - name: Wizard-Vicuna-13B-Uncensored-SpQR
6
+ results:
7
+ - task:
8
+ type: text-generation-inference
9
+ name: Text Generation
10
+ dataset:
11
+ type: c4-new
12
+ name: C4 New Dataset
13
+ metrics:
14
+ - type: perplexity
15
+ value: 7.354
16
+ - task:
17
+ type: text-generation-inference
18
+ name: Text Generation
19
+ dataset:
20
+ type: wikitext2
21
+ name: WikiText-2 Dataset
22
+ metrics:
23
+ - type: perplexity
24
+ value: 5.685
25
+ - task:
26
+ type: text-generation-inference
27
+ name: Text Generation
28
+ dataset:
29
+ type: ptb-new
30
+ name: PTB New Dataset
31
+ metrics:
32
+ - type: perplexity
33
+ value: 20.822
34
  ---
35
+
36
+
37
+ # Wizard-Vicuna-13B-Uncensored-SpQR
38
+
39
+ ## Overview
40
+ This model is an SpQR 4-bit quantization of the original [Wizard-Vicuna-13B-Uncensored-HF](https://huggingface.co/TheBloke/Wizard-Vicuna-13B-Uncensored-HF)
41
+
42
+ ## Quantization Specifications
43
+
44
+ - **Quantization**: 4-bit, group size of 16, per-channel with scale and zero-point of 3 bits.
45
+ - **Outliers**: Threshold set at 0.2.
46
+ - **Permutation Order**: `act_order`.
47
+ - **Dampening**: Set at 1e0.
48
+ - **Sampling**: 128 samples.
49
+ - **Logging**: Via Weights & Biases.
50
+
51
+ ## Evaluation Metrics
52
+
53
+ The following perplexity scores were obtained on various datasets:
54
+
55
+ | Dataset | Perplexity |
56
+ |:---------:|:----------:|
57
+ | c4-new | 7.354 |
58
+ | wikitext2 | 5.685 |
59
+ | ptb-new | 20.822 |