tareq052/soccer-bert

⚽ SOCCER BERT

A multilingual football-specific BERT model trained from scratch.

Model Details

Property	Value
Parameters	159M
Architecture	Decoder-only Transformer
Layers	10
Attention Heads	8
Embedding Dim	512
Context Length	512 tokens
Vocab Size	250,002 (XLM-RoBERTa)
Training Phase	2 of 3
Dataset	3.72 GB football + wikipedia
Format	Safetensors
Status	Private (Phase 3 SFT pending)

Languages

English, Spanish, Portuguese, Arabic, Bengali, Hindi

Citation

@misc{Soccer-BERT-2026,
  author = {Tareq},
  title  = {Football GPT V2},
  year   = {2025},
  url    = {https://huggingface.co/tareq052/football-gpt-v2}
}