Update README.md
Browse files
README.md
CHANGED
@@ -106,8 +106,6 @@ language:
|
|
106 |
|
107 |
**xlm-roberta-longformer** is a multilingual [Longformer](https://arxiv.org/abs/2004.05150) initialized with [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base)'s weights without further pretraining. It is intended to be fine-tuned on a downstream task.
|
108 |
|
109 |
-
The notebook for replicating the model is available on GitHub: https://github.com/hyperonym/dirge/blob/master/models/xlm-roberta-longformer/convert.ipynb
|
110 |
-
|
111 |
| Model | attention_window | hidden_size | num_hidden_layers | model_max_length |
|
112 |
| --- | --- | --- | --- | --- |
|
113 |
| [base](https://huggingface.co/hyperonym/xlm-roberta-longformer-base-16384) | 256 | 768 | 12 | 16384 |
|
|
|
106 |
|
107 |
**xlm-roberta-longformer** is a multilingual [Longformer](https://arxiv.org/abs/2004.05150) initialized with [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base)'s weights without further pretraining. It is intended to be fine-tuned on a downstream task.
|
108 |
|
|
|
|
|
109 |
| Model | attention_window | hidden_size | num_hidden_layers | model_max_length |
|
110 |
| --- | --- | --- | --- | --- |
|
111 |
| [base](https://huggingface.co/hyperonym/xlm-roberta-longformer-base-16384) | 256 | 768 | 12 | 16384 |
|