CausalFocus-GPTBERT-D256-BBPE16K
Hugging Face repository reserved for the champion variant of
causal-focus-gptbert-d256-bbpe16k.
This is a card-only placeholder: model weights are intentionally not uploaded by
the data/artifact release flow. Running bash scripts/upload_release.sh champion
from the submission artifact replaces this card with the full model export.
Expected Model
- repo:
miguelcsx/causal-focus-gptbert-d256-bbpe16k - hidden size: 256
- layers: 4
- attention heads: 4
- feed-forward size: 1024
- context length: 512
- vocabulary size: 16000
- planned words seen: 7000000
Data Artifacts
- corpus:
miguelcsx/babylm-corpus-cleaned - tokenizer:
miguelcsx/causal-focus-bbpe16k-tokenizer - encoded corpus:
miguelcsx/babylm-bbpe16k-512-encoded