Edit model card

luke-japanese-large-lite

luke-japanese is the Japanese version of LUKE (Language Understanding with Knowledge-based Embeddings), a pre-trained knowledge-enhanced contextualized representation of words and entities. LUKE treats words and entities in a given text as independent tokens, and outputs contextualized representations of them. Please refer to our GitHub repository for more details and updates.

This model is a lightweight version which does not contain Wikipedia entity embeddings. Please use the full version for tasks that use Wikipedia entities as inputs.

luke-japaneseは、単語とエンティティの知識拡張型訓練済み Transformer モデルLUKEの日本語版です。LUKE は単語とエンティティを独立したトークンとして扱い、これらの文脈を考慮した表現を出力します。詳細については、GitHub リポジトリを参照してください。

このモデルは、Wikipedia エンティティのエンベディングを含まない軽量版のモデルです。Wikipedia エンティティを入力として使うタスクには、full versionを使用してください。

Experimental results on JGLUE

The experimental results evaluated on the dev set of JGLUE is shown as follows:

Model MARC-ja JSTS JNLI JCommonsenseQA
acc Pearson/Spearman acc acc
LUKE Japanese large 0.965 0.932/0.902 0.927 0.893
Baselines:
Tohoku BERT large 0.955 0.913/0.872 0.900 0.816
Waseda RoBERTa large (seq128) 0.954 0.930/0.896 0.924 0.907
Waseda RoBERTa large (seq512) 0.961 0.926/0.892 0.926 0.891
XLM RoBERTa large 0.964 0.918/0.884 0.919 0.840

The baseline scores are obtained from here.

Citation

@inproceedings{yamada2020luke,
  title={LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention},
  author={Ikuya Yamada and Akari Asai and Hiroyuki Shindo and Hideaki Takeda and Yuji Matsumoto},
  booktitle={EMNLP},
  year={2020}
}
Downloads last month
1,261
Hosted inference API
Mask token: undefined
This model can be loaded on the Inference API on-demand.