README.md

---
language: ti
license: mit
library_name: transformers
tags:
- tigrinya
- gpt2
- text-generation
metrics:
- perplexity
- loss
pipeline_tag: text-generation
---

# Model Card for GPT-2 Tigrinya Medium

## Model Summary
This is a GPT-2 model trained from scratch on Tigrinya text data. It was trained on 20.6 million tokens, primarily from news sources. The model is specifically designed for generating Tigrinya text using the Hugging Face Transformers library.

#### Model Description
- Model type: GPT-2
- Language: Tigrinya (ትግርኛ)
- Finetuned from model: Trained from scratch (no pre-training)

#### Model Architecture
- Parameters: 42.6M
- Context Window: 128 tokens
- **Vocabulary Size:** 52,000

#### Training Details
- Training regime: fp16 mixed precision
- Number of Epochs: 12
- Batch Size: 6 (with gradient accumulation steps of 8)
- Learning Rate: 5e-4

#### Evaluation
- Training Perplexity: 28.6
- Training Loss: 3.12

#### Usage

```python
from transformers import pipeline
# Load the model
generator = pipeline('text-generation', model='luel/gpt2-tigrinya-medium')

prompt = "ክልል ትግራይ"
# Generate text
text = generator(prompt, max_length=100)[0]['generated_text']
print(text)
```

#### Limitations
- Limited context window of 128 tokens.
- Best suited for medium-length Tigrinya text generation.
- Outputs should be reviewed for accuracy.