Stage III dense checkpoints

Raw training checkpoints for the Stage III dense-model experiments.

Repository: DimaTivator/stage3_dense

Configuration	Checkpoint	Iteration	Files
`soap_fp8_states`	final	`75457`	`soap_fp8_states/main.pt`, `soap_fp8_states/worker_0.pt`
`soap_fp8_states`	pre-decay	`67911`	`soap_fp8_states/pre_decay/main.pt`, `soap_fp8_states/pre_decay/worker_0.pt`
`adamw_fp8_states`	final	`75457`	`adamw_fp8_states/main.pt`, `adamw_fp8_states/worker_0.pt`
`adamw_fp8_states`	pre-decay	`67911`	`adamw_fp8_states/pre_decay/main.pt`, `adamw_fp8_states/pre_decay/worker_0.pt`
`muon_fp8_states`	final	`75457`	`muon_fp8_states/main.pt`, `muon_fp8_states/worker_0.pt`
`muon_fp8_states`	pre-decay	`67911`	`muon_fp8_states/pre_decay/main.pt`, `muon_fp8_states/pre_decay/worker_0.pt`
`ademamix_fp8_act`	final	`78269`	`ademamix_fp8_act/main.pt`, `ademamix_fp8_act/worker_0.pt`
`ademamix_fp8_act`	pre-decay	`67911`	`ademamix_fp8_act/pre_decay/main.pt`, `ademamix_fp8_act/pre_decay/worker_0.pt`
`adamw_fp8_act`	final	`78475`	`adamw_fp8_act/main.pt`, `adamw_fp8_act/worker_0.pt`
`adamw_fp8_act`	pre-decay	`67911`	`adamw_fp8_act/pre_decay/main.pt`, `adamw_fp8_act/pre_decay/worker_0.pt`
`muon_fp8_act`	final	`77168`	`muon_fp8_act/main.pt`, `muon_fp8_act/worker_0.pt`
`muon_fp8_act`	pre-decay	`67911`	`muon_fp8_act/pre_decay/main.pt`, `muon_fp8_act/pre_decay/worker_0.pt`

Each configuration contains its final checkpoint at the configuration root and the checkpoint immediately before learning-rate decay in pre_decay/. Every checkpoint contains main.pt and worker_0.pt.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support