You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Weights derived from an unreleased base (Molmo2-4B-SFT). Research use.
Log in or Sign Up to review the conditions and access this model content.
molmo2-codec Stage-1 V4 (trainable E_P P-tokenizer)
Stage-1 pretrained P-tokenizer for the AdaCodec-on-Molmo2 video pipeline, V4.
Unlike V1-V3 (a frozen shared ViT + ~0.49M query params), V4 gives the P-frame path its own trainable ViT branch (E_P, ~381M) — matching AdaCodec's separate ViT branch design (paper Appx E). This was the root cause of the earlier weak codec accuracy (35% vs dense 75%): the P-tokenizer was ~1000x too small to encode video-specific motion.
ptokenizer_step42000.pt— trained ~42k steps on 60k videos @ 2fps. Containsptok.vit_p.*(the trained E_P),ptok.query,ptok.proj,ptok.t_emb, plus the co-trained readout.- Held-out proxy (real vs shuffled vs zeroed P-tokens): video-specific signal grew ~0.007 -> ~0.17.
Config: 2fps sampling, motion search +-48, N_P=16, LR 1e-4 + warmup + cosine.
Code: https://github.com/weikaih04/molmo2-codec (branch adacodec). Loaded into Stage-2 SFT via
CODEC_STAGE1_PTOK in launch_scripts/train_codec_sft.py.