riotu-lab commited on
Commit
ba4a718
1 Parent(s): a3f8bd9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -22
README.md CHANGED
@@ -10,37 +10,34 @@ tags:
10
  # ArabianGPT Model Overview
11
 
12
  ## Introduction
13
- ArabianGPT is a GPT-2 based model, custom-trained for the Arabic language, as part of the ArabianLLM initiatives at Prince Sultan University's Robotics and Internet of Things Lab.
 
 
14
 
15
  ## Key Features
16
-
17
- | Feature | Description |
18
- |---------------------------|----------------------------|
19
- | **Architecture** | GPT-2 |
20
- | **Model Size** | 134 million parameters |
21
- | **Layers** | 12 |
22
- | **Model Attention Layers**| 12 (MAL) |
23
- | **Context Window Size** | 768 tokens |
24
 
25
  ## Training
26
-
27
- | Aspect | Details |
28
- |---------------------|-------------------------------|
29
- | **Dataset** | Abu Elkhiar Corpus |
30
- | **Data Size** | 15.5 GB |
31
- | **Words** | 237.8 million |
32
- | **Tokens** | Over 1.75 billion |
33
- | **Hardware** | NDIVIA A100 |
34
- | **Training Scale** | 7.5 million examples |
35
- | **Training Duration**| 3 days |
36
- | **Performance** | Final loss of 3.97 |
37
 
38
 
39
  ## Role in ArabianLLM Initiatives
40
- ArabianGPT 0.1B is crucial for advancing Arabic language processing, addressing challenges unique to Arabic morphology and dialects.
41
 
42
  ## Usage
43
- Suitable for Arabic text generation tasks. Example usage with Transformers SummarizationPipeline:
44
  ```python
45
  from transformers import pipeline
46
 
 
10
  # ArabianGPT Model Overview
11
 
12
  ## Introduction
13
+ ArabianGPT-0.1B, developed under the ArabianLLM initiatives, is a specialized GPT-2 model optimized for Arabic language modeling.
14
+ It's a product of the collaborative efforts at Prince Sultan University's Robotics and Internet of Things Lab, focusing on enhancing natural language modeling and generation in Arabic.
15
+ This model represents a significant stride in LLM research, specifically addressing the linguistic complexities and nuances of the Arabic language.
16
 
17
  ## Key Features
18
+ - **Architecture**: GPT-2
19
+ - **Model Size**: 134 million parameters
20
+ - **Layers**: 12
21
+ - **Model Attention Layers (MAL)**: 12
22
+ - **Context Window Size**: 768 tokens
 
 
 
23
 
24
  ## Training
25
+ - **Dataset**: Abu Elkhiar Corpus
26
+ - **Data Size**: 15.5 GB
27
+ - **Words**: 237.8 million
28
+ - **Tokenizer**: Aranizer 64K
29
+ - **Tokens**: Over 1.75 billion
30
+ - **Hardware**: 2 NDIVIA A100 GPUs
31
+ - **Training Scale**: 7.5 million examples
32
+ - **Training Duration**: 3 days
33
+ - **Performance**: Final loss of 3.97
 
 
34
 
35
 
36
  ## Role in ArabianLLM Initiatives
37
+ ArabianGPT-0.1B (Base Model) is crucial for advancing Arabic language processing, addressing challenges unique to Arabic morphology and dialects.
38
 
39
  ## Usage
40
+ Suitable for Arabic text generation tasks. Example usage with Transformers Pipeline:
41
  ```python
42
  from transformers import pipeline
43