initial commit

README.md

@@ -0,0 +1,53 @@
+---
+language: 
+- ar
+- dz
+
+tags:
+- pytorch
+- bert
+- multilingual
+- ar
+- dz
+
+license: apache-2.0
+
+widget:
+- text: " أنا من الجزائر من ولاية [MASK] "
+- text: "rabi [MASK] khouya sami"
+- text: " ربي [MASK] خويا لعزيز"
+- text: "tahya el [MASK]."
+- text: "rouhi ya dzayer [MASK]"
+
+inference: true
+---
+
+# DziriBERT
+DziriBERT is the first Transformer-based Language Model that has been pre-trained specifically for the Algerian Dialect. It handles Algerian text contents written using both Arabic and Latin characters. It sets new state of the art results on Algerian text classification datasets, even if it has been pre-trained on much less data (~1 million tweets).
+
+For more information, please visit our paper: link.
+
+## How to use
+
+```python
+from transformers import BertTokenizer, BertForMaskedLM
+
+tokenizer = BertTokenizer.from_pretrained("amine/dziribert")
+model = BertForMaskedLM.from_pretrained("amine/dziribert")
+
+```
+
+### How to cite
+
+```bibtex
+@article{dziribert,
+  title={DziriBERT: a Pre-trained Language Model for the Algerian Dialect},
+  author={Abdaoui, Amine and Berrimi, Mohamed and Oussalah, Mourad and Moussaoui, Abdelouahab},
+  journal={arXiv preprint arXiv:XXX.XXXXX},
+  year={2021}
+}
+```
+
+## Contact 
+
+Please contact amine.abdaoui@huawei.com for any question, feedback or request.

config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": ".",
+  "architectures": [
+    "BertForMaskedLM"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.10.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 50000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0717245dc2d2aa9108bb5266e2541a7df0a11467265e746d4972532e9876d084
+size 498061650

tf_model.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8875357e722fde8db51ab392b59adbfef9ac1694afef4f3ed7ab9523a1d5bd5d
+size 653438424

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "model_max_length": 512, "max_len": 512}