Molmo2 Codec V6 RGB-HEX K4

This repository is the paired V6 RGB-HEX/K4 release for Molmo2 Codec. It is intentionally separate from molmo2-codec-stage2-v6, which contains a historical fast_gray2 diagnostic checkpoint with a different motion and GOP distribution.

Required files

weights/stage1_ptokenizer_step89000.pt: Stage-1 P-tokenizer, P-ViT and P-connector state.
weights/stage2_step1000_consolidated.pt: complete Stage-2 Molmo2 state.
config/stage2_step1000.yaml: training configuration.
config/gamma_v6_rgbhex_k4.json: calibrated GOP threshold and contract.
eval/paired_summary_n200.json: fixed-ID paired Dense/Codec evaluation.
SHA256SUMS: byte identities for every published artifact.

Stage-1 and Stage-2 weights are a pair. Do not combine this Stage-2 model with the old 20K or 17K P-tokenizer checkpoints.

Visual input contract

2 FPS full-span timeline, capped at 2,048 decoded source frames.
Consecutive RGB frames resized to 378 x 378.
14 x 14 RGB block matching with SAD + 0.05 * |motion|_1 in a +/-48-pixel valid-block window.
Large HEXBS search followed by exhaustive 3 x 3 local refinement.
Adaptive GOP with target P=4 and hard maximum P=4.
81 visual tokens per I-frame and 16 per P-frame.
Each P-frame contains signed RGB residual plus (dx, dy) motion.

The motion mode, gamma, GOP profile, FPS, timeline cap and visual budget are part of the cache key. See the GitHub README for cache construction and inference commands.

Current paired evaluation

The fixed-ID evaluation uses 200 examples per benchmark, RGB-HEX K4, an 8,192 visual-token budget and a 16,384-token model sequence for comparability.

benchmark	Dense	Codec K4
MLVU	78.0	78.0
MVBench	76.0	74.5
TempCompass	69.5	64.5
PerceptionTest	82.5	80.0
NExT-QA	83.0	82.0

Video-MME and LVBench are added to eval/paired_summary_n200.json once their same-protocol run completes. The strongest remaining Codec weakness is fine-grained motion direction and speed; this release should not be described as lossless visual reconstruction.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support