EX-AI-Corase-30M
معرفی فارسی
مدل زبانی EX-AI-Corase-30M یک مدل زبانی کوچک مبتنی بر معماری ترنسفورمر علّی (Decoder-only) است که توسط EXAiStudio و به‌عنوان بخشی از پروژه تحقیقاتی EX Corase توسعه یافته است.
این مدل از صفر (From Scratch) و با استفاده از آثار ادبی فردوسی، به‌ویژه شاهنامه، آموزش داده شده است. هدف از توسعه آن، پژوهش در زمینه ساخت و آموزش مدل‌های زبانی کوچک فارسی و بررسی توانایی آن‌ها در پردازش و تولید متن فارسی است.
این مدل دارای 30,425,600 پارامتر است و یک پروژه تحقیقاتی در حوزه مدل‌های زبانی فارسی محسوب می‌شود.
English Introduction
EX-AI-Corase-ShahZaban-30M-2 is a small-scale Persian language model developed by EXAiStudio as part of the EX Corase research project.
The model uses a decoder-only Causal Transformer architecture and was trained from scratch on Ferdowsi's Shahnameh.
With 30,425,600 parameters, this model explores the development and training of compact Persian language models.
- Introduction
EX-AI-Corase-ShahZaban-30M-2 is a small Persian language model based on a Causal Transformer (Decoder-only) architecture, trained entirely from scratch by EXAiStudio.
The model was trained on the Ferdowsi Shahnameh corpus and is part of the EX Corase research project.
- Technical Details
| Parameter | Value |
|---|---|
| Architecture | Causal Transformer (Decoder-only) |
| Parameters | 30,425,600 |
| vocab_size | 10000 |
| hidden_size | 512 |
| num_hidden_layers | 8 |
| num_attention_heads | 8 |
| head_dim | 64 |
| intermediate_size | 1376 |
| max_position_embeddings | 512 |
| Normalization | RMSNorm |
| Positional Encoding | RoPE |
| Activation | SwiGLU |
| tie_word_embeddings | true |
- Training
- Training Method: From Scratch
- Training Corpus: Ferdowsi Shahnameh
- Best Validation Loss: 3.873615
- Model Size
| Precision | Approximate Size |
|---|---|
| FP32 | ~140 MB |
| FP16 | ~70 MB |
| INT8 | ~35 MB |
The approximate model weight sizes are provided in FP32, FP16, and INT8 precision formats.
- Research Project
EX-AI-Corase-ShahZaban-30M-2 is an experimental research model developed as part of EXAiStudio's efforts in small-scale Persian language modeling.
Developed by EXAiStudio