go-inoue commited on
Commit
2f1e3c8
1 Parent(s): 5c9f6ac

Add description for abbreviations

Browse files
Files changed (1) hide show
  1. README.md +7 -6
README.md CHANGED
@@ -11,8 +11,11 @@ widget:
11
  ## Model description
12
 
13
  **CAMeLBERT** is a collection of BERT models pre-trained on Arabic texts with different sizes and variants.
 
 
14
  The details are described in the paper *"[The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://arxiv.org/abs/2103.06678)."*
15
- We release eight models with different sizes and variants as follows:
 
16
 
17
  ||Model|Variant|Size|#Word|
18
  |-|-|:-:|-:|-:|
@@ -25,8 +28,6 @@ We release eight models with different sizes and variants as follows:
25
  ||`bert-base-camelbert-msa-eighth`|MSA|14GB|1.6B|
26
  ||`bert-base-camelbert-msa-sixteenth`|MSA|6GB|746M|
27
 
28
- This model card describes **CAMeLBERT-Mix** (`bert-base-camelbert-mix`), a model pre-trained on a mixture of these variants: CA, DA, and MSA.
29
-
30
  ## Intended uses
31
  You can use the released model for either masked language modeling or next sentence prediction.
32
  However, it is mostly intended to be fine-tuned on an NLP task, such as NER, POS tagging, sentiment analysis, dialect identification, and poetry classification.
@@ -83,15 +84,15 @@ output = model(encoded_input)
83
  ```
84
 
85
  ## Training data
86
- - MSA
87
  - [The Arabic Gigaword Fifth Edition](https://catalog.ldc.upenn.edu/LDC2011T11)
88
  - [Abu El-Khair Corpus](http://www.abuelkhair.net/index.php/en/arabic/abu-el-khair-corpus)
89
  - [OSIAN corpus](https://vlo.clarin.eu/search;jsessionid=31066390B2C9E8C6304845BA79869AC1?1&q=osian)
90
  - [Arabic Wikipedia](https://archive.org/details/arwiki-20190201)
91
  - The unshuffled version of the Arabic [OSCAR corpus](https://oscar-corpus.com/)
92
- - DA
93
  - A collection of dialectal Arabic data described in [our paper](https://arxiv.org/abs/2103.06678).
94
- - CA
95
  - [OpenITI (Version 2020.1.2)](https://zenodo.org/record/3891466#.YEX4-F0zbzc)
96
 
97
  ## Training procedure
 
11
  ## Model description
12
 
13
  **CAMeLBERT** is a collection of BERT models pre-trained on Arabic texts with different sizes and variants.
14
+ We release pre-trained language models for Modern Standard Arabic (MSA), dialectal Arabic (DA), and classical Arabic (CA), in addition to a model pre-trained on a mix of the three.
15
+ We also provide additional models that are pre-trained on a scaled-down set of the MSA variant (half, quarter, eighth, and sixteenth).
16
  The details are described in the paper *"[The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://arxiv.org/abs/2103.06678)."*
17
+
18
+ This model card describes **CAMeLBERT-Mix** (`bert-base-camelbert-mix`), a model pre-trained on a mixture of these variants: MSA, DA, and CA.
19
 
20
  ||Model|Variant|Size|#Word|
21
  |-|-|:-:|-:|-:|
 
28
  ||`bert-base-camelbert-msa-eighth`|MSA|14GB|1.6B|
29
  ||`bert-base-camelbert-msa-sixteenth`|MSA|6GB|746M|
30
 
 
 
31
  ## Intended uses
32
  You can use the released model for either masked language modeling or next sentence prediction.
33
  However, it is mostly intended to be fine-tuned on an NLP task, such as NER, POS tagging, sentiment analysis, dialect identification, and poetry classification.
 
84
  ```
85
 
86
  ## Training data
87
+ - MSA (Modern Standard Arabic)
88
  - [The Arabic Gigaword Fifth Edition](https://catalog.ldc.upenn.edu/LDC2011T11)
89
  - [Abu El-Khair Corpus](http://www.abuelkhair.net/index.php/en/arabic/abu-el-khair-corpus)
90
  - [OSIAN corpus](https://vlo.clarin.eu/search;jsessionid=31066390B2C9E8C6304845BA79869AC1?1&q=osian)
91
  - [Arabic Wikipedia](https://archive.org/details/arwiki-20190201)
92
  - The unshuffled version of the Arabic [OSCAR corpus](https://oscar-corpus.com/)
93
+ - DA (dialectal Arabic)
94
  - A collection of dialectal Arabic data described in [our paper](https://arxiv.org/abs/2103.06678).
95
+ - CA (classical Arabic)
96
  - [OpenITI (Version 2020.1.2)](https://zenodo.org/record/3891466#.YEX4-F0zbzc)
97
 
98
  ## Training procedure