Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,10 @@
|
|
1 |
# MorRoBERTa
|
2 |
-
|
|
|
|
|
|
|
|
|
|
|
3 |
|
4 |
## Usage
|
5 |
The model weights can be loaded using transformers library by HuggingFace.
|
@@ -9,3 +14,13 @@ from transformers import AutoTokenizer, AutoModel
|
|
9 |
tokenizer = AutoTokenizer.from_pretrained("otmangi/MorRoBERTa")
|
10 |
|
11 |
model = AutoModel.from_pretrained("otmangi/MorRoBERTa")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
# MorRoBERTa
|
2 |
+
|
3 |
+
MorRoBERTa is a Transformer-based Language Model designed specifically for the Moroccan Dialect.
|
4 |
+
Developed by Moussaoui Otman and El Younoussi Yacine.
|
5 |
+
|
6 |
+
## About MorRoBERTa
|
7 |
+
MorRoBERTa, designed specifically for the Moroccan dialect, is a scaled-down variant of the RoBERTa-base model. It comprises 6 layers, 12 attention heads, and 768 hidden dimensions. The training process spanned approximately 92 hours, covering 12 epochs on the complete training set. A vast corpus of six million Moroccan dialect sentences, amounting to 71 billion tokens, was used to train this model.
|
8 |
|
9 |
## Usage
|
10 |
The model weights can be loaded using transformers library by HuggingFace.
|
|
|
14 |
tokenizer = AutoTokenizer.from_pretrained("otmangi/MorRoBERTa")
|
15 |
|
16 |
model = AutoModel.from_pretrained("otmangi/MorRoBERTa")
|
17 |
+
|
18 |
+
## Acknowledgments
|
19 |
+
This research was supported through computational resources of HPC-MARWAN (www.marwan.ma/hpc) provided by the National Center for Scientific and Technical Research (CNRST). Rabat. Morocco.
|
20 |
+
|
21 |
+
## Contact
|
22 |
+
For any inquiries, feedback, or requests, please feel free to reach out to :
|
23 |
+
|
24 |
+
otman.moussaoui@etu.uae.ac.ma
|
25 |
+
|
26 |
+
yacine.elyounoussi@uae.ac.ma
|