mmnga commited on
Commit
95e554a
1 Parent(s): 819a51f

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +45 -0
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
4
+
5
+ # llm-jp-13b-instruct-full-dolly-oasst-v1.0-GPTQ-calib-ja-1k
6
+ llm-jpさんが公開している、[llm-jp-13b-instruct-full-dolly-oasst-v1.0](https://huggingface.co/llm-jp/llm-jp-13b-instruct-full-dolly-oasst-v1.0)を、
7
+ 日本語のキャリブレーションセットで生成したGPTQモデルになります。
8
+
9
+ キャリブレーションセットは[izumi-lab/wikipedia-ja-20230720](https://huggingface.co/datasets/izumi-lab/wikipedia-ja-20230720)から、
10
+ 1kほどランダムサンプリングしたものと、
11
+ [ELYZA-tasks-100](https://huggingface.co/datasets/elyza/ELYZA-tasks-100)のinput/outputを計200ほど追加しています。
12
+ [mmnga/wikipedia-ja-20230720-1k](https://huggingface.co/datasets/mmnga/wikipedia-ja-20230720-1k)
13
+
14
+ モデル一覧
15
+ [mmnga/llm-jp-13b-v1.0-4bit-g128-GPTQ-calib-ja-1k](https://huggingface.co/mmnga/llm-jp-13b-v1.0-4bit-g128-GPTQ-calib-ja-1k)
16
+ [mmnga/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0-GPTQ-calib-ja-1k](https://huggingface.co/mmnga/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0-GPTQ-calib-ja-1k)
17
+ [mmnga/llm-jp-13b-instruct-full-dolly-oasst-v1.0-GPTQ-calib-ja-1k](https://huggingface.co/mmnga/llm-jp-13b-instruct-full-dolly-oasst-v1.0-GPTQ-calib-ja-1k)
18
+
19
+ GGUF版
20
+ [mmnga/llm-jp-13b-v1.0-gguf](https://huggingface.co/mmnga/llm-jp-13b-v1.0-gguf)
21
+ [mmnga/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0-gguf](https://huggingface.co/mmnga/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0-gguf)
22
+ [mmnga/llm-jp-13b-instruct-full-dolly-oasst-v1.0-gguf](https://huggingface.co/mmnga/llm-jp-13b-instruct-full-dolly-oasst-v1.0-gguf)
23
+
24
+ # Usage
25
+
26
+ ~~~Bash
27
+ pip install auto-gptq transformers
28
+ ~~~
29
+
30
+ ~~~python
31
+ from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
32
+ from transformers import AutoTokenizer
33
+
34
+ model_name_or_path = "mmnga/llm-jp-13b-instruct-full-jaster-dolly-oasst-v1.0-GPTQ-calib-ja-1k"
35
+
36
+ # Tokenizer
37
+ tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
38
+
39
+ # Model
40
+ model = AutoGPTQForCausalLM.from_quantized(model_name_or_path, use_safetensors=True, device="cuda:0", use_auth_token=False)
41
+
42
+ #Your test prompt
43
+ prompt = """今日の晩御飯のレシピの作り方を教えて ### 回答:"""
44
+ print(tokenizer.decode(model.generate(**tokenizer(prompt, return_tensors="pt",add_special_tokens=False).to(model.device), max_new_tokens=100,do_sample=True,top_p=0.95,temperature=0.7)[0]))
45
+ ~~~