Commit
ยท
276eb8e
1
Parent(s):
cf35f18
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,41 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
+
tags: [gpt2]
|
4 |
+
language: ko
|
5 |
---
|
6 |
+
|
7 |
+
# KoGPT2-small
|
8 |
+
|
9 |
+
| Model | Batch Size | Tokenizer | Vocab Size | Max Length | Parameter Size |
|
10 |
+
|:---: | :------: | :-----: | :------: | :----: | :------: |
|
11 |
+
|GPT2 | 64 | BPE | 30,000 | 1024 | 108M |
|
12 |
+
|
13 |
+
|
14 |
+
# DataSet
|
15 |
+
- AIhub - ์น๋ฐ์ดํฐ ๊ธฐ๋ฐ ํ๊ตญ์ด ๋ง๋ญ์น ๋ฐ์ดํฐ (4.8M)
|
16 |
+
- KoWiki dump 230701 (1.4M)
|
17 |
+
|
18 |
+
|
19 |
+
# Inference Example
|
20 |
+
|
21 |
+
```python
|
22 |
+
from transformers import AutoTokenizer, GPT2LMHeadModel
|
23 |
+
|
24 |
+
text = "์ถ๊ทผ์ด ํ๋ค๋ฉด"
|
25 |
+
|
26 |
+
tokenizer = AutoTokenizer.from_pretrained('Datascience-Lab/GPT2-small')
|
27 |
+
model = GPT2LMHeadModel.from_pretrained('Datascience-Lab/GPT2-small')
|
28 |
+
|
29 |
+
inputs = tokenizer.encode_plus(text, return_tensors='pt', add_special_tokens=False)
|
30 |
+
|
31 |
+
outputs = model.generate(inputs['input_ids'], max_length=128,
|
32 |
+
repetition_penalty=2.0,
|
33 |
+
pad_token_id=tokenizer.pad_token_id,
|
34 |
+
eos_token_id=tokenizer.eos_token_id,
|
35 |
+
bos_token_id=tokenizer.bos_token_id,
|
36 |
+
use_cache=True,
|
37 |
+
temperature = 0.5)
|
38 |
+
outputs = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
39 |
+
|
40 |
+
# ์ถ๋ ฅ ๊ฒฐ๊ณผ : '์ถ๊ทผ์ด ํ๋ค๋ฉด ์ถ๊ทผ์ ํ์ง ์๋ ๊ฒ์ด ์ข๋ค. ํ์ง๋ง ์ถํด๊ทผ ์๊ฐ์ ๋ฆ์ถ๋ ๊ฒ์ ์คํ๋ ค ๊ฑด๊ฐ์ ์ข์ง ์๋ค.. ํนํ๋ ์ฅ์๊ฐ์ ์
๋ฌด๋ก ์ธํด ํผ๋ก๊ฐ ์์ด๊ณ ๋ฉด์ญ๋ ฅ์ด ๋จ์ด์ง๋ฉด, ํผ๋ก๊ฐ์ด ์ฌํด์ ธ์ ์ ๋ค๊ธฐ ์ด๋ ค์ด ๊ฒฝ์ฐ๊ฐ ๋ง๋ค. ์ด๋ฐ ๊ฒฝ์ฐ๋ผ๋ฉด ํ์๋ณด๋ค ๋ ๋ง์ ์์ผ๋ก ๊ณผ์์ ํ๊ฑฐ๋ ๋ฌด๋ฆฌํ ๋ค์ด์ดํธ๋ฅผ ํ ์ ์๋ค. ๋ฐ๋ผ์ ์๋จ ์กฐ์ ๊ณผ ํจ๊ป ์์ ๋ณด์ถฉ์ ์ ๊ฒฝ ์จ์ผ ํ๋ค. ๋ํ ๊ณผ๋ํ ์์์ด ์ฒด์ค ๊ฐ๋์ ๋์์ ์ฃผ๋ฏ๋ก ์ ์ ํ ์ด๋๋์ ์ ์งํ๋ ๊ฒ๋ ์ค์ํ๋ค.'
|
41 |
+
```
|