K024 commited on
Commit
353c83d
1 Parent(s): 87490a4

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +31 -0
README.md ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - zh
4
+ - en
5
+ tags:
6
+ - glm
7
+ - chatglm
8
+ - thudm
9
+ ---
10
+
11
+ # ChatGLM2 6b int8 量化模型
12
+
13
+ 详情参考 [K024/chatglm-q](https://github.com/K024/chatglm-q)。
14
+
15
+ See [K024/chatglm-q](https://github.com/K024/chatglm-q) for more details.
16
+
17
+ ```python
18
+ import torch
19
+ from chatglm_q.decoder import ChatGLMDecoder, chat_template
20
+
21
+ device = torch.device("cuda")
22
+ decoder = ChatGLMDecoder.from_pretrained("K024/chatglm2-6b-int8", device=device)
23
+
24
+ prompt = chat_template([], "我是谁?")
25
+ for text in decoder.generate(prompt):
26
+ print(text)
27
+ ```
28
+
29
+ 模型权重按 ChatGLM2-6b 许可发布,见 [MODEL LICENSE](https://huggingface.co/THUDM/chatglm2-6b/blob/main/MODEL_LICENSE)。
30
+
31
+ Model weights are released under the same license as ChatGLM2-6b, see [MODEL LICENSE](https://huggingface.co/THUDM/chatglm2-6b/blob/main/MODEL_LICENSE).