This is RoBERTa model pretrained on texts in the Japanese language.
3.45GB wikipedia text
trained 1.65M step
use the sentencepiece tokenizer.
If you want to fine-tune model. Please use
from transformers import BertTokenizer, RobertaModel
BertTokenizer.from_pretrained('')
RoBERTModel.from_pretrained('')
The accuracy in JGLUE-marc_ja-v1.0 binary sentiment classification 95.4%
Contribute by Yokohama Nationaly University Mori Lab
@article{liu2019roberta,
title={Roberta: A robustly optimized bert pretraining approach},
author={Liu, Yinhan and Ott, Myle and Goyal, Naman and Du, Jingfei and Joshi, Mandar and Chen, Danqi and Levy, Omer and Lewis, Mike and Zettlemoyer, Luke and Stoyanov,
Veselin},
journal={arXiv preprint arXiv:1907.11692},
year={2019}
}
- Downloads last month
- 2