Tiny DeepSeek3 Village Generator
A tiny character-level language model based on the DeepSeek3 architecture for generating Turkish village names.
This model was trained from scratch for educational purposes using 11,104 unique Turkish village names.
Model Information
- Architecture: DeepSeek3 (Tiny MoE)
- Parameters: 49,064 (≈36,776 active per token)
- Vocabulary: 64 characters
- Context Length: 32
- Training Steps: 8,000
Dataset
Training dataset:
- Uunan/turkish-village-names
The dataset contains one Turkish village name per line.
Example Output
Hamamza
Yeniçen
Arslangazi
Dosmancı
İzdelik
Demirağı
Ahlatören
Sofulu
Aşağıkaraköy
Akçi
Usage
python generate.py
Notes
- Character-level language model
- Trained from scratch
- Uses a tiny Mixture-of-Experts (MoE) architecture inspired by DeepSeek3
- Intended for educational and research purposes
- Outputs are synthetic and may not correspond to real village names
Acknowledgements
This project is based on the educational repository: