metadata
language:
- multilingual
- ar
- bn
- de
- el
- en
- es
- fi
- fr
- hi
- id
- it
- ja
- ko
- nl
- pl
- pt
- ru
- sv
- sw
- te
- th
- tr
- vi
- zh
thumbnail: https://github.com/studio-ousia/luke/raw/master/resources/luke_logo.png
tags:
- luke
- named entity recognition
- relation classification
- question answering
license: apache-2.0
mLUKE
mLUKE (multilingual LUKE) is a multilingual extension of LUKE.
Please check the official repository for more details and updates.
This is the mLUKE base model with 12 hidden layers, 768 hidden size. The total number of parameters in this model is 561M. The model was initialized with the weights of XLM-RoBERTa(large) and trained using December 2020 version of Wikipedia in 24 languages.
This model is a lite-weight version of studio-ousia/mluke-large, without Wikipedia entity embeddings but only with special entities such as [MASK].
Citation
If you find mLUKE useful for your work, please cite the following paper:
@inproceedings{ri-etal-2022-mluke,
title = "m{LUKE}: {T}he Power of Entity Representations in Multilingual Pretrained Language Models",
author = "Ri, Ryokan and
Yamada, Ikuya and
Tsuruoka, Yoshimasa",
booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
year = "2022",
url = "https://aclanthology.org/2022.acl-long.505",