LR-AttnRes-n4
0.5B Block Attention Residuals checkpoint trained on 9,999,745,024 tokens.
- Attention-residual blocks (
n):4 - Routing rank (
r):1024 - Reported final validation loss:
2.9797 - Final checkpoint step:
38146 - Training tokens processed:
9999745024 - Artifact:
final_model.pt - Artifact size:
2472703487bytes - Artifact LFS SHA256:
a6267abc23ed6a860bcdcc125dc204ef290d70de436758115272c56d9d241726
The validation value is the owner-recorded full-sweep result. The original upload did not include a structured evaluation JSON.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support