ISLAM-PO/masrygpt-flash

AraGPT2-Fresh-Forgotten Model

Overview

aragpt2-fresh-forgotten is an Arabic language model built on the GPT-2 Small architecture by OpenAI, trained for Arabic text generation.

• Type: GPT2LMHeadModel (causal language modeling)
• Language: Arabic (with support for Latin characters, digits, and punctuation)
• Size: 135M parameters (515 MB in safetensors FP32 format)
• Vocabulary: 64,000 tokens (ByteLevel-BPE)
• Context length: 1024 tokens

Folder Contents

File	Approx. Size	Description
config.json	881 bytes	Architecture and hyperparameter settings
generation_config.json	204 bytes	Default generation settings
model.safetensors	~515 MB	Model weights
tokenizer.json	~4.6 MB	Tokenizer (BPE)
vocab.json	~1.9 MB	Vocabulary (64k)
merges.txt	~1.5 MB	Merge rules (63,740 merges)
README.md	—	This document
ARCHITECTURE.md	—	Detailed architecture description
USAGE.md	—	Usage guide with code examples
TOKENIZER.md	—	Tokenizer and vocabulary guide

Quick Specs (from config.json)

{
  "model_type": "gpt2",
  "architectures": ["GPT2LMHeadModel"],
  "n_layer": 12,
  "n_head": 12,
  "n_embd": 768,
  "n_positions": 1024,
  "n_ctx": 1024,
  "vocab_size": 64000,
  "activation_function": "gelu_new",
  "bos_token_id": 0,
  "eos_token_id": 0
}

Quickstart

from transformers import GPT2LMHeadModel, GPT2Tokenizer

model_path = r"C:\Users\RDP\Desktop\aragpt2-fresh-forgotten"
tokenizer = GPT2Tokenizer.from_pretrained(model_path)
model = GPT2LMHeadModel.from_pretrained(model_path)

prompt = "Artificial intelligence in the Arabic language"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_length=100,
    do_sample=True,
    top_p=0.95,
    num_beams=5,
    repetition_penalty=3.0,
    no_repeat_ngram_size=3
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

For full details see: USAGE.md, ARCHITECTURE.md, TOKENIZER.md

Intended Uses

• Arabic text generation (stories, articles, sentence completion).
• Research and experimentation on Arabic language modeling.
• Fine-tuning for tasks such as dialogue, summarization, and generative classification.

Limitations and Cautions

1. Hallucination: May generate incorrect information — verify facts independently.
2. Limited context: Only 1024 tokens (~700–900 Arabic words approx.).
3. Small size: Only 12 layers — less capable than modern large models.
4. Bias: Reflects biases in the Arabic training data.
5. Start/end tokens: bos_token_id = eos_token_id = 0 (<|endoftext|>) — take care during fine-tuning.

License and Source

• Architecture: GPT-2 (OpenAI).
• Check the original training-data license before commercial use.

---

Automatically documented from the model files on 2026-09-03.