seonglae
/

llama-2-7b-chat-hf-gptq

Text Generation

text-generation-inference

Model card Files Files and versions Community

seonglae commited on Jul 19, 2023

Commit

a73d561

•

1 Parent(s): 148fced

Update README.md

Files changed (1) hide show

README.md +3 -0

README.md CHANGED Viewed

@@ -9,10 +9,13 @@ tags:
 - 7b
 - llama
 - 4bit
 ---
 # Get Started
 This model should use [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) so you need to use `auto-gptq`
 ```py
 from transformers import AutoTokenizer, pipeline, LlamaForCausalLM, LlamaTokenizer

 - 7b
 - llama
 - 4bit
+- quantization
 ---
 # Get Started
 This model should use [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) so you need to use `auto-gptq`
+- `no-act-order` model
+- 4bit model quantization
 ```py
 from transformers import AutoTokenizer, pipeline, LlamaForCausalLM, LlamaTokenizer