File size: 1,763 Bytes
44b735e 85f3628 44b735e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
---
base_model: CausalLM/14B
datasets:
- JosephusCheung/GuanacoDataset
- Open-Orca/OpenOrca
- stingning/ultrachat
- meta-math/MetaMathQA
- liuhaotian/LLaVA-Instruct-150K
- jondurbin/airoboros-3.1
- WizardLM/WizardLM_evol_instruct_V2_196k
- RyokoAI/ShareGPT52K
- RyokoAI/Fandom23K
- milashkaarshif/MoeGirlPedia_wikitext_raw_archive
- wikipedia
- wiki_lingua
- fnlp/moss-003-sft-data
- garage-bAInd/Open-Platypus
- LDJnr/Puffin
- openbmb/llava_zh
- BAAI/COIG
- TigerResearch/tigerbot-zhihu-zh-10k
- liwu/MNBVC
- teknium/openhermes
inference: false
language:
- en
- zh
license: wtfpl
model_creator: CausalLM
model_name: CausalLM 14B
model_type: llama
pipeline_tag: text-generation
prompt_template: '<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
'
quantized_by: cgus
tags:
- llama
- llama2
---
# CausalLM 14B - GPTQ
- Model creator: [CausalLM](https://huggingface.co/CausalLM)
- Original model: [CausalLM 14B](https://huggingface.co/CausalLM/14B)
<!-- description start -->
## Description
[4bpw h6](https://huggingface.co/cgus/CausalLM-14B-exl2/tree/main)
[3.5bpw h6](https://huggingface.co/cgus/CausalLM-14B-exl2/tree/3.5bpw-h6)
Experimental exl2 quantization for CausalLM-14B for Exllamav2.
I had some issues during quantization process, so I suspect it might have quality issues.
3.5bpw version barely fits 12GB VRAM but has unusually high perplexity for wikitext dataset.
I couldn't measure perplexity for 4bpw version and to compare it with TheBloke's GPTQ, so I have no idea if my quantization has issues or it supposed to be like this.
You could try this exl2 version but I'd recommend to use [TheBloke's GPTQ](https://huggingface.co/TheBloke/CausalLM-14B-GPTQ) version instead. |