temporal-moe-router-adapt

Router adaptation checkpoints for the Temporal-MoE rolling-residency router, 56 files, 2.80 GiB.

These are the routers produced by the OLMoE adaptation experiments, where a rolling-residency router is fitted onto a pretrained OLMoE backbone. They are safetensors files, not full models. Loading one requires the base model, allenai/OLMoE-1B-7B-0924, which is not mirrored here.

Layout

adapt_ckpts/     13 router safetensors from the bake and sweep experiments
                 router_bake_{B,C,CE,D,E,Er8,Er64,G,H}.safetensors
                 router_cal2.safetensors
                 router_sweep_lr{3e-5,1e-4,3e-4}.safetensors
olmoe_adapt/     3 additional routers, throughput and validation runs
                 router_tputCkpt16, router_tputMB4, router_validate
metadata/        bakeoff_*.json and train_*.json config dumps, bake_*.log and sweep_*.log
                 training logs, one per router

The two file-size families are not an error. The small routers (about 4 MB) store router weights only. The large ones (122 MB to 944 MB) store expanded router parameterizations from the Er8, Er64, E, CE, G and H variants.

Reading these

Each router has a matching bakeoff_*.json in metadata/ recording the configuration it was trained with, and a bake_*.log or sweep_*.log with the training trace. A router without its config is not interpretable, so both are included.

The full optimizer and training state behind the CE, G, H, Er8, Er64 and D bakes is in ncylich/temporal-moe-extras under olmoe_adapt/ckpt_bake_*.pt.

MANIFEST.csv and the cited column

A manifest covering every file in all four repositories lives in the code repository. It has seven columns: local_path, hf_repo, hf_path, bytes, sha256, run_name, cited.

The cited column marks whether a run is referenced by a results table in results/ablations/*.csv or by the paper:

• cited, the run backs a published number. There are 58 of these.
• uncited, the run is infrastructure validation, a smoke test, a throughput probe, or an aborted run. It is kept for completeness and reproducibility, not because a table depends on it. There are 13 of these.
• empty, the file is not scoped to a single run, for example a batch log or an evaluation output.

Every sha256 in the manifest was computed on this disk before upload and each file was verified to exist remotely with a matching byte size.

Links

• Code: https://github.com/ncylich/temporal-moe
• Paper: Temporal-MoE (short paper), see the paper/ directory in the code repository
• Upstream platform this work forks: FLAME-MoE, arXiv:2505.20225

Companion repositories

• ncylich/temporal-moe-ckpts, Megatron training checkpoints
• ncylich/temporal-moe-router-adapt, router adaptation safetensors
• ncylich/temporal-moe-extras, captures, merged model, result tables, figures
• ncylich/temporal-moe-corpus, tokenized training corpus

Provenance

Trained with a personal fork of FLAME-MoE. Not affiliated with or endorsed by the FLAME-MoE authors.