README.md

---
datasets:
- AfnanTS/Final_ArLAMA_DS_tokenized_for_ARBERTv2
language:
- ar
base_model:
- UBC-NLP/ARBERTv2
pipeline_tag: fill-mask
---


<img src="./arab_icon2.png" alt="Model Logo" width="30%" height="30%" align="right"/>

**ARBERTv2_ArLAMA** is a transformer-based Arabic language model fine-tuned on Masked Language Modeling (MLM) tasks. The model uses Knowledge Graphs (KGs) to enhance its understanding of semantic relations and improve its performance in various Arabic NLP tasks.



## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use


Filling masked tokens in Arabic text, particularly in contexts enriched with knowledge from KGs.


### Downstream Use

Can be further fine-tuned for Arabic NLP tasks that require semantic understanding, such as text classification or question answering.



## How to Get Started with the Model

```python
from transformers import pipeline
fill_mask = pipeline("fill-mask", model="AfnanTS/ARBERTv2_ArLAMA")
fill_mask("اللغة [MASK] مهمة جدا."
```

## Training Details

### Training Data

Trained on the ArLAMA dataset, which is designed to represent Knowledge Graphs in natural language.



### Training Procedure

Continued pre-training of ArBERTv2 using Masked Language Modeling (MLM) tasks, integrating structured knowledge from Knowledge Graphs.