elmadany commited on
Commit
823a383
1 Parent(s): 1b5e0d8

commit from elmadany

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -1,12 +1,12 @@
1
 
2
  <img src="https://raw.githubusercontent.com/UBC-NLP/marbert/main/ARBERT_MARBERT.jpg" alt="drawing" width="30%" height="30%" align="right"/>
3
 
4
- **ARBERT** is one of three models described in our ACl 2021 paper ["ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic"](https://mageed.arts.ubc.ca/files/2020/12/marbert_arxiv_2020.pdf). ARBERT is a large-scale pre-trained masked language model focused on Modern Standard Arabic (MSA). To train ARBERT, we use the same architecture as BERT-base: 12 attention layers, each has 12 attention heads and 768 hidden dimensions, a vocabulary of 100K WordPieces, making ∼163M parameters. We train ARBERT on a collection of Arabic datasets comprising **61GB of text** (**6.2B tokens**). For more information, please visit our own GitHub [repo](https://github.com/UBC-NLP/marbert).
5
 
6
 
7
  # BibTex
8
 
9
- If you use our models (ARBERT, MARBERT, or MARBERT_V2) for your scientific publication, or if you find the resources in this repository useful, please cite our paper as follows (to be updated):
10
  ```bibtex
11
  @inproceedings{abdul-mageed-etal-2021-arbert,
12
  title = "{ARBERT} {\&} {MARBERT}: Deep Bidirectional Transformers for {A}rabic",
 
1
 
2
  <img src="https://raw.githubusercontent.com/UBC-NLP/marbert/main/ARBERT_MARBERT.jpg" alt="drawing" width="30%" height="30%" align="right"/>
3
 
4
+ **ARBERT** is one of three models described in our **ACl 2021 paper** **["ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic"](https://mageed.arts.ubc.ca/files/2020/12/marbert_arxiv_2020.pdf)**. ARBERT is a large-scale pre-trained masked language model focused on Modern Standard Arabic (MSA). To train ARBERT, we use the same architecture as BERT-base: 12 attention layers, each has 12 attention heads and 768 hidden dimensions, a vocabulary of 100K WordPieces, making ∼163M parameters. We train ARBERT on a collection of Arabic datasets comprising **61GB of text** (**6.2B tokens**). For more information, please visit our own GitHub [repo](https://github.com/UBC-NLP/marbert).
5
 
6
 
7
  # BibTex
8
 
9
+ If you use our models (ARBERT, MARBERT, or MARBERTv2) for your scientific publication, or if you find the resources in this repository useful, please cite our paper as follows (to be updated):
10
  ```bibtex
11
  @inproceedings{abdul-mageed-etal-2021-arbert,
12
  title = "{ARBERT} {\&} {MARBERT}: Deep Bidirectional Transformers for {A}rabic",