weikaih/molmo2-codec-stage2-v6

Molmo2 Codec V6 — Stage-2 step 100

This is the evaluated V6 AdaCodec-on-Molmo2 research checkpoint. It packages the Stage-2 model, its matching Stage-1 P-tokenizer, the fixed motion-cost calibration, the exact saved training config, source provenance, and a paired six-benchmark evaluation summary.

This is not a standalone transformers checkpoint. It must be loaded with the Molmo2 Codec code and its lmms-eval adapter.

Checkpoints

File	Purpose	SHA256
weights/stage2_step100_consolidated.pt	Consolidated Molmo2 + trained Stage-2 parameters	141467de9a28a53d6ebb87ef8bcf1dbf191880d39b64dd0bebc3728baaf79677
weights/stage1_ptokenizer_step20000.pt	V6+HEX Stage-1 P-tokenizer	10fc7d0460199581d9a7f1677c79a6a519adc55184c4ba14cd44f50a4bf5eec9

Training contract

• Base: Molmo2-4B-SFT lineage.
• Stage-1: V6 P-tokenizer, RGB HEX motion search, step 20,000.
• Stage-2 checkpoint: step 100 from the formal 32,768-token run.
• Training visual ceiling: 16,384 tokens; text ceiling: 15,872 tokens.
• Candidate timeline: full-span sampling, 2 FPS, at most 2,048 source frames.
• Mixture: 40% long codec caption, 10% normal codec video, 37.5% native video, and 12.5% native image replay.
• Stage-2 trains the LLM, P-ViT, and P-connector. The I-frame visual path stays on the native Molmo2 representation.
• The durable Stage-2 GOP cache uses the explicit fast_gray2 motion-search contract and its matching gamma artifact. The packaged Stage-1 weights were trained with RGB HEX; this run intentionally adapts the P branch in Stage-2.

The complete serialized configuration is in config/stage2_step100_config.yaml.

Paired evaluation

Each row uses the same frozen 200 examples for Dense and Codec. Evaluation uses full-span 2 FPS sampling, a 2,048-frame candidate timeline, 8,192 visual tokens, and a 16,384-token sequence. The checked machine-readable result is eval/paired_summary_n200.json.

Benchmark	Dense	Codec	Codec − Dense
Video-MME	70.0	69.0	-1.0
LVBench	48.0	49.5	+1.5
MVBench	75.5	68.5	-7.0
TempCompass	69.0	68.5	-0.5
PerceptionTest	82.0	83.0	+1.0
NExT-QA	84.0	83.5	-0.5

The checkpoint is close to Dense on five of six paired subsets, but the MVBench regression is material and remains an open limitation. These are bounded n=200 diagnostics, not full benchmark submissions.

Code and loading

• Main code: https://github.com/weikaih04/molmo2-codec, branch adacodec.
• Main release commits: 69f20d99, 63874827, ead01f2e.
• lmms-eval adapter commit: 6fda918 in the local Molmo2 evaluation fork.
• Example launcher: examples/molmo2_codec/run_smoke.sh in that fork.

The adapter expects paths to both packaged weight files and the gamma artifact. It invokes the production Molmo2 preprocessing, timestamp, position, packing, GOP-selection, and codec inference code instead of reimplementing them.

Provenance

provenance/SOURCE_SNAPSHOT.txt records the exact source state captured by the formal training job, including the pre-commit working-tree digest and the post-requeue decode fix. provenance/SOURCE_FILE_SHA256SUMS records hashes of the evaluation-critical source files. The three release commits above organize that working tree and add the final regression tests; they should not be mistaken for a claim that the model was retrained after code cleanup.

Limitations

• Research checkpoint after only 100 Stage-2 optimizer steps.
• Requires the custom code path; it is not directly loadable through AutoModel.from_pretrained.
• The evaluation sample size is 200 per benchmark.
• Motion cache and gamma artifacts are contract-bound. Do not silently mix RGB HEX and fast_gray2 cache namespaces.
• The upstream base-model license and usage restrictions continue to apply.