Edit model card


luke-japanese is the Japanese version of LUKE (Language Understanding with Knowledge-based Embeddings), a pre-trained knowledge-enhanced contextualized representation of words and entities. LUKE treats words and entities in a given text as independent tokens, and outputs contextualized representations of them. Please refer to our GitHub repository for more details and updates.

This model is a lightweight version which does not contain Wikipedia entity embeddings. Please use the full version for tasks that use Wikipedia entities as inputs.

luke-japaneseは、単語とエンティティの知識拡張型訓練済み Transformer モデルLUKEの日本語版です。LUKE は単語とエンティティを独立したトークンとして扱い、これらの文脈を考慮した表現を出力します。詳細については、GitHub リポジトリを参照してください。

このモデルは、Wikipedia エンティティのエンベディングを含まない軽量版のモデルです。Wikipedia エンティティを入力として使うタスクには、full versionを使用してください。

Experimental results on JGLUE

The experimental results evaluated on the dev set of JGLUE is shown as follows:

Model MARC-ja JSTS JNLI JCommonsenseQA
acc Pearson/Spearman acc acc
LUKE Japanese large 0.965 0.932/0.902 0.927 0.893
Tohoku BERT large 0.955 0.913/0.872 0.900 0.816
Waseda RoBERTa large (seq128) 0.954 0.930/0.896 0.924 0.907
Waseda RoBERTa large (seq512) 0.961 0.926/0.892 0.926 0.891
XLM RoBERTa large 0.964 0.918/0.884 0.919 0.840

The baseline scores are obtained from here.


  title={LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention},
  author={Ikuya Yamada and Akari Asai and Hiroyuki Shindo and Hideaki Takeda and Yuji Matsumoto},
Downloads last month
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.