|
--- |
|
license: apache-2.0 |
|
|
|
widget: |
|
- text: 横浜国立大学は日本の[MASK]奈川県にある。 |
|
--- |
|
|
|
|
|
This is RoBERTa model pretrained on texts in the Japanese language. |
|
|
|
3.45GB wikipedia text |
|
|
|
trained 125M step |
|
|
|
use the BERT BPE tokenizer. |
|
|
|
If you want to fine-tune model. Please use |
|
```python |
|
|
|
|
|
from transformers import BertTokenizer, RoFormerModel |
|
BertTokenizer.from_pretrained('Roformer-base-japanese') |
|
RoFormerModel.from_pretrained('Roformer-base-japanese') |
|
``` |
|
|
|
|
|
The accuracy in JGLUE-marc-ja-v1.0 binary sentiment classification 95.12% |
|
|
|
|
|
|
|
|
|
Contribute by Yokohama Nationaly University Mori Lab |
|
|
|
@article{su2021roformer, |
|
title={Roformer: Enhanced transformer with rotary position embedding}, |
|
author={Su, Jianlin and Lu, Yu and Pan, Shengfeng and Wen, Bo and Liu, Yunfeng}, |
|
journal={arXiv preprint arXiv:2104.09864}, |
|
year={2021} |
|
} |
