nishimura999 commited on
Commit
d2ee97b
1 Parent(s): ce2e9aa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +91 -1
README.md CHANGED
@@ -8,7 +8,9 @@ tags:
8
  - trl
9
  license: apache-2.0
10
  language:
11
- - en
 
 
12
  ---
13
 
14
  # Uploaded model
@@ -20,3 +22,91 @@ language:
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  - trl
9
  license: apache-2.0
10
  language:
11
+ - ja
12
+ datasets:
13
+ - kinokokoro/ichikara-instruction-003
14
  ---
15
 
16
  # Uploaded model
 
22
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
23
 
24
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
25
+
26
+
27
+ # usage
28
+ ## -import
29
+ ```python
30
+ # 必要なライブラリを読み込み
31
+ from unsloth import FastLanguageModel
32
+ from peft import PeftModel
33
+ import torch
34
+ import json
35
+ from tqdm import tqdm
36
+ import re
37
+ ```
38
+
39
+ ## -setting
40
+ ```python
41
+ # Hugging Faceで取得したToken
42
+ HF_TOKEN = "{Your hugging face token}"
43
+
44
+ # モデルのIDと、LoRAのアダプタ名
45
+ model_id = "llm-jp/llm-jp-3-13b"
46
+ adapter_id = "nishimura999/llm-jp-3-13b-it-v102_lora"
47
+ ```
48
+
49
+ ## -load
50
+ ```python
51
+ # unslothのFastLanguageModelで元のモデルをロード。
52
+ dtype = None
53
+ load_in_4bit = True
54
+ model, tokenizer = FastLanguageModel.from_pretrained(
55
+ model_name=model_id,
56
+ dtype=dtype,
57
+ load_in_4bit=load_in_4bit,
58
+ trust_remote_code=True,
59
+ )
60
+ # 元のモデルにLoRAのアダプタを統合。
61
+ model = PeftModel.from_pretrained(model, adapter_id, token = HF_TOKEN)
62
+ ```
63
+
64
+ ## -dataset
65
+ ```python
66
+ # データセットの読み込み。
67
+ datasets = []
68
+ with open("./elyza-tasks-100-TV_0.jsonl", "r") as f:
69
+ item = ""
70
+ for line in f:
71
+ line = line.strip()
72
+ item += line
73
+ if item.endswith("}"):
74
+ datasets.append(json.loads(item))
75
+ item = ""
76
+ ```
77
+
78
+ ## -generate
79
+ ```python
80
+ # モデルを用いてタスクの推論。
81
+
82
+ FastLanguageModel.for_inference(model)
83
+
84
+ results = []
85
+ for dt in tqdm(datasets):
86
+ input = dt["input"]
87
+
88
+ prompt = f"""### 指示\n{input}\n### 回答\n"""
89
+
90
+ inputs = tokenizer([prompt], return_tensors = "pt").to(model.device)
91
+
92
+ outputs = model.generate(**inputs, max_new_tokens = 512, use_cache = True, do_sample=False, repetition_penalty=1.2)
93
+ prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).split('\n### 回答')[-1]
94
+
95
+ results.append({"task_id": dt["task_id"], "input": input, "output": prediction})
96
+ ```
97
+
98
+ ## -output
99
+ ```python
100
+ # 結果をjsonlで保存。
101
+ json_file_id = re.sub(".*/", "", adapter_id)
102
+ with open(f"./{json_file_id}_output.jsonl", 'w', encoding='utf-8') as f:
103
+ for result in results:
104
+ json.dump(result, f, ensure_ascii=False)
105
+ f.write('\n')
106
+ ```
107
+
108
+ # ref
109
+ ### 本モデルは下記のデータを使ってファインチューニングしております。ここでデータ提供者に感謝申し上げます。
110
+ (https://liat-aip.sakura.ne.jp/wp/llmのための日本語インストラクションデータ作成/llmのための日本語インストラクションデータ-公開/)
111
+ 関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎.
112
+ ichikara-instruction: LLMのための日本語インストラクションデータの構築. 言語処理学会第30回年次大会(2024)