Edit model card

deberta-base-japanese-wikipedia

Model Description

This is a DeBERTa(V2) model pre-trained on Japanese Wikipedia and 青空文庫 texts. NVIDIA A100-SXM4-40GB took 109 hours 27 minutes for training. You can fine-tune deberta-base-japanese-wikipedia for downstream tasks, such as POS-tagging, dependency-parsing, and so on.

How to Use

from transformers import AutoTokenizer,AutoModelForMaskedLM
tokenizer=AutoTokenizer.from_pretrained("KoichiYasuoka/deberta-base-japanese-wikipedia")
model=AutoModelForMaskedLM.from_pretrained("KoichiYasuoka/deberta-base-japanese-wikipedia")

Reference

安岡孝一: 青空文庫DeBERTaモデルによる国語研長単位係り受け解析, 東洋学へのコンピュータ利用, 第35回研究セミナー (2022年7月), pp.29-43.

Downloads last month
12
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.