Go Inoue commited on
Commit
e77cb23
1 Parent(s): f522b66

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -4
README.md CHANGED
@@ -6,12 +6,12 @@ widget:
6
  - text: "الهدف من الحياة هو [MASK] ."
7
  ---
8
 
9
- # CAMeLBERT-MSA
10
 
11
  ## Model description
12
 
13
- **CAMeLBERT** is a BERT model pre-trained on Arabic texts with different sizes and variants.
14
- The details are described in the paper *"The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models."*
15
  We release eight models with different sizes and variants as follows:
16
 
17
  ||Model|Variant|Size|#Word|
@@ -25,7 +25,7 @@ We release eight models with different sizes and variants as follows:
25
  ||`bert-base-camelbert-msa-eighth`|MSA|14GB|1.6B|
26
  ||`bert-base-camelbert-msa-sixteenth`|MSA|6GB|746M|
27
 
28
- This model card describes `bert-base-camelbert-msa`, a model pre-trained on the entire MSA dataset.
29
 
30
  ## Intended uses
31
  You can use the released model for either masked language modeling or next sentence prediction.
@@ -57,6 +57,8 @@ You can use this model directly with a pipeline for masked language modeling:
57
  'token_str': 'الحب'}]
58
  ```
59
 
 
 
60
  Here is how to use this model to get the features of a given text in PyTorch:
61
  ```python
62
  from transformers import AutoTokenizer, AutoModel
 
6
  - text: "الهدف من الحياة هو [MASK] ."
7
  ---
8
 
9
+ # CAMeLBERT: A collection of pre-trained models for Arabic NLP tasks
10
 
11
  ## Model description
12
 
13
+ **CAMeLBERT** is a collection of BERT models pre-trained on Arabic texts with different sizes and variants.
14
+ The details are described in the paper *"[The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://arxiv.org/abs/2103.06678)."*
15
  We release eight models with different sizes and variants as follows:
16
 
17
  ||Model|Variant|Size|#Word|
 
25
  ||`bert-base-camelbert-msa-eighth`|MSA|14GB|1.6B|
26
  ||`bert-base-camelbert-msa-sixteenth`|MSA|6GB|746M|
27
 
28
+ This model card describes **CAMeLBERT-MSA** (`bert-base-camelbert-msa`), a model pre-trained on the entire MSA dataset.
29
 
30
  ## Intended uses
31
  You can use the released model for either masked language modeling or next sentence prediction.
 
57
  'token_str': 'الحب'}]
58
  ```
59
 
60
+ *Note*: to download our models, you would need `transformers>=3.5.0`. Otherwise, you could download the models manually.
61
+
62
  Here is how to use this model to get the features of a given text in PyTorch:
63
  ```python
64
  from transformers import AutoTokenizer, AutoModel