You need to agree to share your Hugging Face information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
The NativeCodec repositories use manual approval. Weights, configs, tokenizers and dataset archives are available after your access request is approved. Please describe your intended use; the model card and dataset card remain readable without approval.
Log in or Sign Up to review the conditions and access this model content.
NativeCodec-4B-PredLen
4B video MLLM aligned with NativeCodec's learned variable-length token predictor.
This is the predicted-length variant of NativeCodec-4B: the language model is fine-tuned to consume visual tokens produced by the NativeCodec incremental encoder, and each P frame's suffix length is chosen at inference by the learned length predictor. Use it together with the codec checkpoints and code in Tennineee/NativeCodec.
| Companion asset | Repository |
|---|---|
| Code, incremental encoder, prediction heads | Tennineee/NativeCodec |
| Fixed-ratio aligner for the same codec | Tennineee/NativeCodec-4B-FixedRatio |
| BallBench dataset | Tennineee/NativeCodec-BallBench |
Access request
This repository is public but gated with manual approval. Anonymous visitors can read this model card and see the file list. Downloading files requires approval.
To request access, open the repository page, sign in, click Agree and access repository / Request access, and submit the form. Access is granted to individual users, not to organizations. Requests are reviewed manually by the maintainers without an automatic approval or a published turnaround guarantee; the Hub notifies you when a decision is made. A rejected request cannot be resubmitted from the same account on the Hub.
After approval, download the files with an authenticated session (the full model is about 9.34 GB / 8.70 GiB):
hf auth login
hf download Tennineee/NativeCodec-4B-PredLen --local-dir NativeCodec-4B-PredLen
hf download Tennineee/NativeCodec --local-dir NativeCodec
The codec checkpoints required at inference (weights/inc_enc_stage1_step18750.pt and one weights/pred_len_*.pt) live in the code repository. The complete card of Tennineee/NativeCodec also covers access requests, mirror downloads and fine-grained token settings.
Files
Every file sits at the repository root, so the directory can be passed to from_pretrained directly.
| File | Size (bytes) | Content |
|---|---|---|
model.safetensors |
9,319,821,272 | Qwen3.5-4B language model, aligned to NativeCodec tokens; the adapter is merged into the base weights (no separate LoRA checkpoint) |
config.json |
3,161 | Qwen3.5-4B model config (architecture Qwen3_5ForConditionalGeneration) |
preprocessor_config.json |
390 | image processor config, inherited from the base model |
video_preprocessor_config.json |
385 | video processor config, inherited from the base model |
tokenizer.json |
12,807,982 | tokenizer (vocabulary 248,320) |
tokenizer_config.json |
16,710 | tokenizer settings, including the chat template reference |
vocab.json |
6,722,759 | vocabulary |
merges.txt |
3,353,259 | BPE merges |
chat_template.jinja |
7,756 | chat template |
LICENSE-APACHE, NOTICE.md, README.md |
license, attribution and this card |
The weights are a complete Qwen3.5-4B-sized model: about 4.66 B parameters in bfloat16 (the file size divided by two bytes), including the frozen visual tower inherited from the base model. NativeCodec does not route video through that visual tower; the codec front-end adds its own visual tokens to the language model input.
Loading
The checkpoint follows the Hugging Face Qwen3.5 layout, so the standard multimodal loader works once your environment provides Qwen3.5 support (the upstream config records transformers_version: 4.57.0.dev0):
from transformers import AutoModelForImageTextToText, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Tennineee/NativeCodec-4B-PredLen")
model = AutoModelForImageTextToText.from_pretrained(
"Tennineee/NativeCodec-4B-PredLen",
torch_dtype="bfloat16",
attn_implementation="flash_attention_2",
)
Loading this directory with vanilla transformers gives the aligned multimodal language model itself. Video reasoning through NativeCodec additionally needs the incremental encoder, the length predictor and the codec front-end:
import sys
sys.path.insert(0, "NativeCodec/stage3_llm_sft")
from modeling_recodec_qwen35 import ReCodecQwen35ForConditionalGeneration
model = ReCodecQwen35ForConditionalGeneration(
model_name_or_path="NativeCodec-4B-PredLen",
recodec_root="NativeCodec/stage3_llm_sft/recodec",
phase1_ckpt="NativeCodec/weights/inc_enc_stage1_step18750.pt",
phase2_ckpt="NativeCodec/weights/pred_len_delta08_step3150.pt",
video_pad_token_id=video_pad_id, # from the code repository's Stage 3 config
max_gop_frames=64,
max_frames=12288,
max_video_tokens=64000,
fixed_p_ratio=None, # learned length predictor
lora_cfg=None, # adapter already merged
)
A complete runnable path, including frame sampling and prompt assembly, is provided by stage3_llm_sft/demo_recodec_inference.py in the code repository.
Two length predictors
The companion code repository ships two Stage 2 length predictors that share one architecture and are interchangeable:
| Checkpoint | Teacher tolerance | Behaviour |
|---|---|---|
weights/pred_len_delta00_step9000.pt |
delta = 0.0 dB | stricter reconstruction, more tokens per P frame |
weights/pred_len_delta08_step3150.pt |
delta = 0.8 dB | about 45% fewer tokens than delta = 0.0, recommended default |
The released checkpoints and results use the delta = 0.8 predictor unless stated otherwise.
Use, budget and results
On BallBench at 16 fps the aligned model reaches 28.2% accuracy with predicted lengths, the highest of the open-source models compared. On TimeLens-Bench with a 64k visual-token budget the predicted-length variant reaches 54.4 mIoU (TimeLens-Bench all) and the constant 1/16 suffix ratio reaches 56.4 mIoU, against 50.7 for the Qwen3.5-4B base. The full comparison and the token-budget ablations are in the code repository's model card.
The visual-token budget is configured by the codec front-end, not by these weights; max_video_tokens in your config decides how many tokens the model sees.
License and attribution
The fine-tuned weights are distributed under the Apache License, Version 2.0, following the license of Qwen/Qwen3.5-4B-Base. The full license text is in LICENSE-APACHE; NOTICE.md records the upstream attribution and the changes made here. The code in Tennineee/NativeCodec is released under the MIT License.
NativeCodec Team
- Downloads last month
- 29
Model tree for Tennineee/NativeCodec-4B-PredLen
Base model
Qwen/Qwen3.5-4B-Base