LR-AttnRes-n16
0.5B Block Attention Residuals checkpoint trained on 9,999,745,024 tokens.
- Attention-residual blocks (
n):16 - Routing rank (
r):1024 - Reported final validation loss:
2.9673 - Final checkpoint step:
38146 - Training tokens processed:
9999745024 - Artifact:
final_model.pt - Artifact size:
2472703487bytes - Artifact LFS SHA256:
c7990c09f78714893513617dd291829cef5278602a7cd979620933fa1e146296
The validation value is the owner-recorded full-sweep result. The original upload did not include a structured evaluation JSON.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support