|
--- |
|
language: |
|
- multilingual |
|
- ar |
|
- bn |
|
- de |
|
- el |
|
- en |
|
- es |
|
- fi |
|
- fr |
|
- hi |
|
- id |
|
- it |
|
- ja |
|
- ko |
|
- nl |
|
- pl |
|
- pt |
|
- ru |
|
- sv |
|
- sw |
|
- te |
|
- th |
|
- tr |
|
- vi |
|
- zh |
|
thumbnail: https://github.com/studio-ousia/luke/raw/master/resources/luke_logo.png |
|
tags: |
|
- luke |
|
- named entity recognition |
|
- relation classification |
|
- question answering |
|
license: apache-2.0 |
|
--- |
|
|
|
## mLUKE |
|
|
|
**mLUKE** (multilingual LUKE) is a multilingual extension of LUKE. |
|
|
|
Please check the [official repository](https://github.com/studio-ousia/luke) for |
|
more details and updates. |
|
|
|
This is the mLUKE base model with 12 hidden layers, 768 hidden size. The total number |
|
of parameters in this model is 561M. |
|
The model was initialized with the weights of XLM-RoBERTa(large) and trained using December 2020 version of Wikipedia in 24 languages. |
|
|
|
This model is a lite-weight version of [studio-ousia/mluke-large](https://huggingface.co/studio-ousia/mluke-large), without Wikipedia entity embeddings but only with special entities such as `[MASK]`. |
|
|
|
### Citation |
|
|
|
If you find mLUKE useful for your work, please cite the following paper: |
|
|
|
```latex |
|
@inproceedings{ri-etal-2022-mluke, |
|
title = "m{LUKE}: {T}he Power of Entity Representations in Multilingual Pretrained Language Models", |
|
author = "Ri, Ryokan and |
|
Yamada, Ikuya and |
|
Tsuruoka, Yoshimasa", |
|
booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)", |
|
year = "2022", |
|
url = "https://aclanthology.org/2022.acl-long.505", |
|
``` |
|
|
