j0no12/nero-optimizer-work-bad

Nero Optimizer Work — BAD (Brutally Adaptive Descent)

This repository contains the final MLX checkpoint for the BAD (Brutally Adaptive Descent) arm of the Nero Optimizer Work research sweep.

Summary

• Optimizer: bad
• Requested learning rate: the run's configured default/controller schedule
• Training budget: 500,000,000 tokens
• Context length: 128 tokens
• Backend: Apple MLX
• Model family: matched dense-deep decoder, 2,048-token vocabulary, 128-wide residual stream, 6 blocks, 32-dimensional attention heads, 148-wide gated MLP
• Stored model parameters: approximately 999,680
• Final training loss: 3.320012
• Final logged throughput: 370,794 tokens/s
• Tail throughput: 370,735 tokens/s (median of the final logged samples)
• Final tokens seen: 500,000,000

Files

• model.npz — final MLX model weights from checkpoint_000500000000
• state.json — checkpoint state metadata
• run.json — frozen run configuration
• metrics.jsonl — complete training metrics log
• config.json — model and publication metadata

Evaluation

The figures above are training-run measurements. An independent held-out validation artifact was not saved with these runs, so this card does not claim a validation score. Compare checkpoints using the same frozen evaluation pass before drawing quality conclusions.

Intended use and limitations

This is an experimental research checkpoint, not an instruction-tuned or production-ready language model. It is published to make the optimizer comparison reproducible. The raw MLX weights require a compatible local MLX loader and are not a Transformers checkpoint.

Reproducibility

All arms use the same prepared finephrase-balanced-500m-2k-v2 token stream, 128-token context, 32-example batches, and 500M-token target. The full local training log is included in metrics.jsonl.

License

No new model license is asserted by this experimental publication. Review the source-data terms before redistribution or downstream use.