LR-AttnRes-n8
0.5B Block Attention Residuals checkpoint trained on 9,999,745,024 tokens.
- Attention-residual blocks (
n):8 - Routing rank (
r):1024 - Reported final validation loss:
2.9778 - Final checkpoint step:
38146 - Training tokens processed:
9999745024 - Artifact:
final_model.pt - Artifact size:
2472703487bytes - Artifact LFS SHA256:
e07c31d44e777808c951fa109953427229887c9278c876dfd6278bd497022d20
The validation value is the owner-recorded full-sweep result. The original upload did not include a structured evaluation JSON.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support