t5-v1_1-base-ko / README.md
hyunwoo3235's picture
Update README.md
e115d45
metadata
language: ko
license: apache-2.0

hyunwoo3235/t5-v1_1-base-ko

Google's T5 Version 1.1 that trained on korean corpus

t5-v1_1-base-ko์€ ํ•œ๊ตญ์–ด ์ฝ”ํผ์Šค์—์„œ ํ•™์Šต๋œ t5 v1.1 ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค.

OOV์„ ๋ง‰๊ธฐ ์œ„ํ•ด BBPE๋ฅผ ์‚ฌ์šฉํ•˜์˜€์œผ๋ฉฐ, HyperCLOVA์—์„œ ํ˜•ํƒœ์†Œ ๋ถ„์„์ด ์„ฑ๋Šฅ์„ ๋†’ํžˆ๋Š”๋ฐ ๋„์›€์ด ๋˜๋Š” ๊ฒƒ์„ ๋ณด๊ณ  ํ† ํฌ๋‚˜์ด์ € ํ•™์Šต ๊ณผ์ •์—์„œ MeCab์„ ์ด์šฉํ•ด ํ˜ˆํƒœ์†Œ๊ฐ€ ์ด์ƒํ•˜๊ฒŒ ํ† ํฐํ™” ๋˜์ง€ ์•Š๋„๋ก ํ•˜์˜€์Šต๋‹ˆ๋‹ค.

Usage

from transformers import AutoTokenizer, T5ForConditionalGeneration

tokenizer = AutoTokenizer.from_pretrained('hyunwoo3235/t5-v1_1-base-ko')
model = T5ForConditionalGeneration.from_pretrained('hyunwoo3235/t5-v1_1-base-ko')