Agron — domain continued pretraining (1 epoch)
Continued from teddyoas/Agron for one full epoch
over the documented code + cybersecurity + WikiText replay corpus.
Held-out evaluation
| Source | Before | After | Delta |
|---|---|---|---|
| code | 1.453 | 1.245 | -0.208 |
| cyber | 1.358 | 1.064 | -0.294 |
| replay | 5.626 | 5.024 | -0.602 |
Training
| Field | Value |
|---|---|
| Steps | 7246 |
| Tokens | 118718464 |
| Block size | 512 |
| Batch / grad accum | 16 x 2 |
| LR / warmup | 0.0003 / 100 |
| Epochs | 1.0 |
| Wall time | 3659.7 s |
| Hardware | 1x Tesla T4 (fp16) |
Generated by watch_agron.py; full metrics in report.json and eval_history.json.
- Downloads last month
- -
Model tree for teddyoas/Agron-epoch1
Base model
teddyoas/Agron