Migrate model card from transformers-repo
Browse filesRead announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/allenai/longformer-base-4096-extra.pos.embd.only/README.md
README.md
ADDED
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
|
2 |
+
# longformer-base-4096-extra.pos.embd.only
|
3 |
+
|
4 |
+
This model is similar to `longformer-base-4096` but it was pretrained to preserve RoBERTa weights by freezing all RoBERTa weights and only train the additional position embeddings.
|
5 |
+
|
6 |
+
|
7 |
+
### Citing
|
8 |
+
|
9 |
+
If you use `Longformer` in your research, please cite [Longformer: The Long-Document Transformer](https://arxiv.org/abs/2004.05150).
|
10 |
+
```
|
11 |
+
@article{Beltagy2020Longformer,
|
12 |
+
title={Longformer: The Long-Document Transformer},
|
13 |
+
author={Iz Beltagy and Matthew E. Peters and Arman Cohan},
|
14 |
+
journal={arXiv:2004.05150},
|
15 |
+
year={2020},
|
16 |
+
}
|
17 |
+
```
|
18 |
+
|
19 |
+
`Longformer` is an open-source project developed by [the Allen Institute for Artificial Intelligence (AI2)](http://www.allenai.org).
|
20 |
+
AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.
|