Commit
β’
63e8784
1
Parent(s):
eb8fdd0
Update README.md
Browse files
README.md
CHANGED
@@ -36,20 +36,79 @@ korean multi-task instruction dataset
|
|
36 |
- CUDA Version: 12.2
|
37 |
|
38 |
## Training
|
39 |
-
Refer github
|
40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
41 |
## Evaluation
|
42 |
|
43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
44 |
|
45 |
-
| model | score | average score | % |
|
46 |
-
| ------------------------------ | ------- |---------------|------------|
|
47 |
-
| gpt-3.5-turbo | 147 | 3.97 | 79.45% |
|
48 |
-
| WizardLM-13B-V1.2 | 96 | 2.59 | 51.89% |
|
49 |
-
| Llama-2-7b-chat-hf | 67 | 1.81 | 36.21% |
|
50 |
-
| Llama-2-13b-chat-hf | 73 | 1.91 | 38.37% |
|
51 |
-
| **komt-llama2-7b-v1 (ours)** | **117** | **3.16** | **63.24%** |
|
52 |
-
| **komt-llama2-13b-v1 (ours)** | **129** | **3.48** | **69.72%** |
|
53 |
|
54 |
------------------------------------------------
|
55 |
# Original model card: Meta's Llama 2 7B-chat
|
|
|
36 |
- CUDA Version: 12.2
|
37 |
|
38 |
## Training
|
39 |
+
Refer https://github.com/davidkim205/komt
|
40 |
+
|
41 |
+
## Usage
|
42 |
+
```
|
43 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
44 |
+
from transformers import TextStreamer, GenerationConfig
|
45 |
+
|
46 |
+
model_name='davidkim205/komt-llama2-7b-v1'
|
47 |
+
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
|
48 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
49 |
+
streamer = TextStreamer(tokenizer)
|
50 |
+
|
51 |
+
def gen(x):
|
52 |
+
generation_config = GenerationConfig(
|
53 |
+
temperature=0.8,
|
54 |
+
top_p=0.8,
|
55 |
+
top_k=100,
|
56 |
+
max_new_tokens=512,
|
57 |
+
early_stopping=True,
|
58 |
+
do_sample=True,
|
59 |
+
)
|
60 |
+
q = f"### instruction: {x}\n\n### Response: "
|
61 |
+
gened = model.generate(
|
62 |
+
**tokenizer(
|
63 |
+
q,
|
64 |
+
return_tensors='pt',
|
65 |
+
return_token_type_ids=False
|
66 |
+
).to('cuda'),
|
67 |
+
generation_config=generation_config,
|
68 |
+
pad_token_id=tokenizer.eos_token_id,
|
69 |
+
eos_token_id=tokenizer.eos_token_id,
|
70 |
+
streamer=streamer,
|
71 |
+
)
|
72 |
+
result_str = tokenizer.decode(gened[0])
|
73 |
+
|
74 |
+
start_tag = f"\n\n### Response: "
|
75 |
+
start_index = result_str.find(start_tag)
|
76 |
+
|
77 |
+
if start_index != -1:
|
78 |
+
result_str = result_str[start_index + len(start_tag):].strip()
|
79 |
+
return result_str
|
80 |
+
|
81 |
+
print(gen('μ μ£Όλλ₯Ό 1λ°2μΌλ‘ νΌμ μ¬ννλ €κ³ νλλ° μ¬ν μ½μ€λ₯Ό λ§λ€μ΄μ€'))
|
82 |
+
```
|
83 |
+
output
|
84 |
+
```
|
85 |
+
### Response: μ μ£Όλλ₯Ό 1λ°2μΌλ‘ νΌμ μ¬ννλ €λ©΄ λ€μκ³Ό κ°μ μ¬ν μ½μ€λ₯Ό λ§λ€μ΄ κ³νν μ μμ΅λλ€:
|
86 |
+
|
87 |
+
1μΌμ°¨:
|
88 |
+
- μμΉ¨: μ μ£Όλμ μλ¦λ€μ΄ ν΄λ³μ ꡬ경νκΈ° μν΄ ν΄λ³μ λμ°©νμΈμ. μΌμΆμ κ°μνλ©° μμ°μ μλ¦λ€μμ λ§λ½νμΈμ.
|
89 |
+
- μ€ν: μ μ£Όλμ λνμ μΈ κ΄κ΄μ§μΈ νλΌμ°μ νννμΈμ. λ±μ°λ‘λ₯Ό λ°λΌ μ¬λΌκ°λ©΄μ κ²½μΉλ₯Ό μ¦κΈ°κ³ μ€λͺ
μ λ£μΌλ©° μ¬μ΄ μ°μ±
μ μ¦κΈ°μΈμ.
|
90 |
+
- μ λ
: μ μ£Όλμ λ§μλ μμμ μμ μ λ
μ 보λ΄μΈμ. μ μ ν ν΄μ°λ¬Όκ³Ό ν₯μ λ£λ‘ λ§λ μμμ λ§λ³΄λ κ²μ μ μ£Όλ μ¬νμ μλ²½ν κ²½νμ΄ λ κ²μ
λλ€.
|
91 |
+
|
92 |
+
2μΌμ°¨:
|
93 |
+
- μμΉ¨: νλΌμ° μΌλλ₯Ό νννκΈ° μν΄ νλΌμ° μΌμ΄νλ‘ μ΄λνμΈμ. μ΄ μΌμ΄νλ λ±μ°μ μ¦κΈ°λ μ¬λλ€μκ² μ΅μ μ μ νμ
λλ€.
|
94 |
+
|
95 |
+
```
|
96 |
## Evaluation
|
97 |
|
98 |
+
| model | score | average(0~5) | percentage |
|
99 |
+
| --------------------------------------- | ------- | ------------ | ---------- |
|
100 |
+
| gpt-3.5-turbo(close) | 147 | 3.97 | 79.45% |
|
101 |
+
| naver Cue(close) | 140 | 3.78 | 75.67% |
|
102 |
+
| clova X(close) | 136 | 3.67 | 73.51% |
|
103 |
+
| WizardLM-13B-V1.2(open) | 96 | 2.59 | 51.89% |
|
104 |
+
| Llama-2-7b-chat-hf(open) | 67 | 1.81 | 36.21% |
|
105 |
+
| Llama-2-13b-chat-hf(open) | 73 | 1.91 | 38.37% |
|
106 |
+
| nlpai-lab/kullm-polyglot-12.8b-v2(open) | 70 | 1.89 | 37.83% |
|
107 |
+
| kfkas/Llama-2-ko-7b-Chat(open) | 96 | 2.59 | 51.89% |
|
108 |
+
| beomi/KoAlpaca-Polyglot-12.8B(open) | 100 | 2.70 | 54.05% |
|
109 |
+
| **komt-llama2-7b-v1 (open)(ours)** | **117** | **3.16** | **63.24%** |
|
110 |
+
| **komt-llama2-13b-v1 (open)(ours)** | **129** | **3.48** | **69.72%** |
|
111 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
112 |
|
113 |
------------------------------------------------
|
114 |
# Original model card: Meta's Llama 2 7B-chat
|