NOESIS-MiniCPM-V-4.6-SigLIP2-BF16 (inside NOESIS-3.5B-A0.5B-DUBBING-FILM)

Vision-only BF16 extraction derivative of openbmb/MiniCPM-V-4.6 — the SigLIP2 vision tower (453 tensors, 996 MB) extracted from MiniCPM-V-4.6 and rebuilt as a stand-alone HF SiglipVisionModel-compatible bundle. Integrated as the vision submodule for Stage 4 / 6 / 16 of the parent NOESIS-3.5B-A0.5B-DUBBING-FILM dubbing pipeline.

✅ APACHE 2.0 — COMMERCIAL USE PERMITTED. Both upstreams are Apache 2.0: OpenBMB MiniCPM-V-4.6 (vision_tower weights extracted here) and Google's SigLIP2 (the original vision-encoder design). This NOESIS extraction inherits the same Apache 2.0 terms and is commercial-safe inside the parent NOESIS-3.5B bundle.

NOESIS / AMAImedia

Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO — Deterministic Hybrid Control Framework for Frozen Neural Operators).

• Founder: Ilia Bolotnikov
• Organization: AMAImedia.com
• X (Twitter): @AMAImediacom
• LinkedIn: Ilia Bolotnikov
• Telegram: @djbionicl
• NOESIS version: v15.9
• Extraction date: 2026-05
• Last updated: 2026-08-28

---

Use case

Vision-only auxiliary encoder inside the NOESIS-3.5B-A0.5B-DUBBING-FILM dubbing pipeline. The encoder consumes face / mouth crops or scene frames and produces dense patch-level visual features [B, num_patches, 1152] consumed by three pipeline stages:

NOESIS pipeline stage	What this encoder does
Stage 4 — Active speaker detection	Visual feature stream complements the audio-only Sortformer/TitaNet diarization; resolves who is currently speaking on screen when audio is ambiguous (overlap, off-screen voices, background music)
Stage 6 — Forced-alignment refinement	Lip-motion cues from the vision tower tighten word-boundary timestamps produced by the audio-only forced aligner (Qwen3-ForcedAligner / MMS-FA), typically improving precision by ±10 ms on close-up shots
Stage 16 — Lip-sync QC	After dubbing rendering, the encoder verifies that the new dubbed audio still aligns with the original lip movement on the output video — gating final delivery

Production NOESIS routes vision through this Apache-2.0 stack (MiniCPM-V-4.6 SigLIP2 → AMAImedia extraction) — no CC-BY-NC or GPLv3 vision-encoder mixing is allowed by the parent NOESIS-3.5B sealed rule set (see R-APACHE-CLEAN, R-NO-VISION-NC below).

Property	Value
Immediate parent	openbmb/MiniCPM-V-4.6
Backbone foundation	SigLIP2 (Google, Apache 2.0)
Extracted prefix	model.vision_tower.* (453 tensors)
HF-compatible class	SiglipVisionModel
Hidden size	1152
Hidden layers	27
Attention heads	16
Intermediate size	4304
Patch size	14
Image size	980 (high-res variant used inside MiniCPM-V-4.6)
Mel / channels	3 (RGB)
Activation	gelu_pytorch_tanh
LayerNorm eps	1e-6
Format	BF16 (native, single safetensors file + .pt state_dict alias)
Bundle size on disk	996.3 MB
Estimated VRAM (BF16 inference)	~0.97 GB (FP16/BF16) · ~0.49 GB INT8 if quantized
License	Apache 2.0 (commercial-ok)

NOESIS-3.5B-A0.5B-DUBBING-FILM vision stack — where this fits

The parent NOESIS-3.5B bundle uses a vision-only auxiliary encoder for Stage 4/6/16 work. This SigLIP2 extraction is the canonical pick:

Component	License	Format	Use case	Status
MiniCPM-V-4.6 SigLIP2 BF16 ← you are here	Apache 2.0	BF16	vision-only encoder (active speaker / lip motion / lip QC)	✅ chosen
MiniCPM-V-4.6 full MLLM	Apache 2.0	BF16	full vision-language (not needed — NOESIS uses own Qwopus3.5-0.8B-Reasoning LLM)	❌ discarded
Google siglip2-so400m-patch14-384 raw	Apache 2.0	BF16	base SigLIP2 without MiniCPM fine-tune	⚠️ less robust on lip/face crops

Why MiniCPM-V-4.6's SigLIP2 vs raw Google SigLIP2: MiniCPM-V fine-tunes its vision tower extensively on multi-image / dense-caption / video data during MLLM training. The resulting tower carries OCR + grounded detection + temporal cues that raw Google SigLIP2 lacks. For lip-motion forced alignment on movie-style close-up shots, the MiniCPM-tuned tower outperforms the raw base by a clear margin while staying Apache 2.0.

What's in this bundle

File	Description
model.safetensors	SigLIP2 vision_tower weights, BF16, 453 tensors, 996.3 MB
siglip2_vision_encoder.pt	Same weights as torch state_dict (for direct torch.load)
config.json	SiglipVisionModel-compatible config (rebuilt from MiniCPM-V-4.6 vision_config block)
preprocessor_config.json	Image preprocessor (resize/normalize) — original from MiniCPM
chat_template.jinja	Preserved upstream (unused by vision-only inference)
extraction_manifest.json	Provenance + extraction metadata (NOESIS layer)
original_README.md	Original openbmb/MiniCPM-V-4.6 README (preserved for reference)
LICENSE	Apache 2.0 + NOESIS attribution banner
README.md	This file
_extraction.log	Extraction-time stdout log (debug)

What was DISCARDED

• LLM backbone — Qwen3.5-0.8B hybrid inside MiniCPM-V-4.6. NOESIS uses its own Qwopus3.5-0.8B-Reasoning trunk for text/LM, so the embedded MiniCPM LM is redundant.
• Multimodal projection — the image_features → LLM embedding linear adapter. NOESIS connects raw SigLIP2 patch features through its own projection head (separate, trained for dubbing-specific tasks).
• Chat templates / tokenizer — vision-only inference does not use them.
• Generation config — vision-only inference has no autoregressive decoding loop.

The extraction reduces 4.5+ GB MiniCPM-V-4.6 weights down to 996 MB of pure vision-tower BF16 tensors.

Why BF16 (not NF4)

This bundle is intentionally kept BF16, not NF4:

Reason	Detail
Already fits	~0.97 GB VRAM peak — comfortable on 6 GB RTX 3060 alongside any other 4-GB-class NOESIS expert
Already fast	Vision-encoder forward is one-shot per frame; FlashAttention-2 keeps RTF well under 0.05× on consumer GPU
Quality preservation	SigLIP2 patch embeddings are downstream-consumed by lip-sync regression heads — NF4 noise on the encoder leaks into ±5 ms timing jitter on Stage 6
Pipeline scheduling	Sequential-swap pattern: vision encoder loads + runs + unloads per scene, so the 1 GB BF16 cost does not stack with other experts

If a strictly smaller footprint is needed for embedded inference, bitsandbytes 4-bit quantization can be applied post-load (see quantization_config=BitsAndBytesConfig(load_in_4bit=True, ...) in HF docs); this is not the default path inside NOESIS-3.5B.

Quick start

Loading the SigLIP2 encoder (NOESIS path)

import torch
from transformers import SiglipVisionModel, SiglipVisionConfig

bundle = r"B:\...\NOESIS-3.5B-A0.5B-DUBBING-FILM\NOESIS-MiniCPM-V-4.6-SigLIP2-BF16"

config = SiglipVisionConfig.from_pretrained(bundle)
model = SiglipVisionModel(config)

# Option A — load from safetensors (recommended, mmap-friendly)
from safetensors.torch import load_file
state = load_file(f"{bundle}/model.safetensors")
model.load_state_dict(state, strict=True)

# Option B — load from torch .pt state_dict (alternative)
# state = torch.load(f"{bundle}/siglip2_vision_encoder.pt", map_location="cpu")
# model.load_state_dict(state, strict=True)

model = model.to(torch.bfloat16).cuda().eval()

# Inference: pixel_values is [B, 3, 980, 980] BF16 on CUDA
with torch.no_grad():
    out = model(pixel_values)
    image_features = out.last_hidden_state          # [B, num_patches, 1152]
    pooled = out.pooler_output if out.pooler_output is not None else image_features.mean(1)

Using upstream preprocessor

from transformers import AutoImageProcessor

processor = AutoImageProcessor.from_pretrained(bundle)
inputs = processor(images=pil_image, return_tensors="pt").to("cuda", torch.bfloat16)
features = model(**inputs).last_hidden_state

Sealed rules (NOESIS DHCF-FNO)

• R-APACHE-CLEAN — Apache 2.0 preserved end-to-end. Vendored copy may be freely combined with other Apache-2.0 NOESIS components (Qwen3-ASR-0.6B / 1.7B, omniASR-CTC, Qwen3-ForcedAligner, parent NOESIS-3.5B trunk).
• R-VENDORED-INTERNAL — this folder is an internal vendor copy inside the parent NOESIS-3.5B-A0.5B-DUBBING-FILM bundle, not a separately distributed NOESIS publication. Distribution happens as part of the parent NOESIS-3.5B archive.
• R-VISION-ONLY-EXTRACT — only model.vision_tower.* was extracted. LLM backbone, multimodal projection, chat templates and tokenizer are intentionally discarded. Re-introducing them inside this bundle is forbidden — the parent NOESIS-3.5B trunk owns the language path (Qwopus3.5).
• R-BF16-NATIVE — keep as upstream BF16; do not NF4-quantize this bundle by default (quality regression on ±10 ms lip-sync timing outweighs the ~0.5 GB VRAM savings on 6 GB GPU).
• R-FROZEN-VISION-OPERATOR — SigLIP2 vision tower is treated as a frozen neural operator under DHCF-FNO. No gradient backprop through these weights inside Phase 1 NOESIS training (scope-locked to LoRA r≤8 on the parent NOESIS-3.5B trunk only).
• R-DUBBING-FILM-SCOPE — vision encoder serves film-dubbing tasks (Stage 4 active speaker + Stage 6 lip-motion forced alignment + Stage 16 lip-sync QC). Not for general image captioning, OCR, or generic VQA — those pathways are out of NOESIS Phase 1 scope (SCOPE LOCK 2026-05-15).
• R-NO-VISION-NC — production NOESIS vision must remain on Apache 2.0 (this bundle). Mixing CC-BY-NC / non-commercial vision encoders (e.g. MMS-style NC variants) into the dubbing pipeline is forbidden by R-APACHE-CLEAN propagation.
• R-NO-FROM-SCRATCH-VLM — per Phase 1 scope lock, no training of custom vision-language models from scratch. This SigLIP2 tower is used as-is.
• R-NEVER-DELETE-WITHOUT-EXPLICIT-CONSENT — MUST NOT be deleted without explicit operator instruction "удали

Provenance

Step	Source / output
Upstream foundation	Google SigLIP2 (Apache 2.0)
Immediate parent	openbmb/MiniCPM-V-4.6 (Apache 2.0) — full MLLM with SigLIP2 vision_tower + Qwen3.5-0.8B LM + projection
Source path on disk	D:\models\llm\MiniCPM-V-4.6
Extraction method	direct safetensors prefix filter on model.vision_tower.* (bypassed AutoModel because MiniCPM-V-4.6 requires transformers>=5.7.0 but NOESIS env is locked to 5.6.0.dev0 for Qwopus3.5 compatibility — see CLAUDE.md GOLDEN RULE 4)
Extraction script	scripts/extract_minicpm_siglip2_v2.py
Recipe doc	noesis_omni_tts/NOESIS_DOWNLOAD_BUILD_PLAN_v3.md §S4.1
Tensors extracted	453
Bytes total	1 044 598 624 (996.3 MB)
Dtype	BF16 (preserved from upstream)
Discarded	LLM backbone (qwen3_5 hybrid 0.8B), multimodal projection, chat templates, tokenizer, generation config
HF-compat target	SiglipVisionModel (config rebuilt from MiniCPM vision_config block)
Extraction wall time	2.9 sec
NOESIS version	v15.9, 2026-05-17 (last updated 2026-05-27)

Citation

@proceedings{yu2025minicpmv45cookingefficient,
  title  = {MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture,
            Data, and Training Recipe},
  author = {Tianyu Yu and Zefan Wang and Chongyi Wang and Fuwei Huang and
            Wenshuo Ma and Zhihui He and Tianchi Cai and Weize Chen and
            Yuxiang Huang and Yuanqian Zhao and others},
  year   = {2025},
  url    = {https://arxiv.org/abs/2509.18154}
}

@article{yao2024minicpm,
  title  = {MiniCPM-V: A GPT-4V Level MLLM on Your Phone},
  author = {Yao, Yuan and Yu, Tianyu and Zhang, Ao and Wang, Chongyi and
            Cui, Junbo and Zhu, Hongji and Cai, Tianchi and Li, Haoyu and
            Zhao, Weilin and He, Zhihui and others},
  journal= {arXiv preprint arXiv:2408.01800},
  year   = {2024}
}

@misc{noesis2026minicpmsiglip2,
  title  = {NOESIS DHCF-FNO :: MiniCPM-V-4.6 SigLIP2 vision-only extraction
            (BF16, 996 MB) for film-dubbing Stage 4/6/16},
  author = {Bolotnikov, Ilia and AMAImedia},
  year   = {2026},
  note   = {Vision-only BF16 extraction derivative of openbmb/MiniCPM-V-4.6
            rebuilt as stand-alone SiglipVisionModel for active-speaker
            detection, lip-motion forced alignment, and lip-sync QC inside
            the NOESIS-3.5B-A0.5B-DUBBING-FILM dubbing pipeline.},
  url    = {https://amaimedia.com}
}

License

Apache License 2.0. Original MiniCPM-V-4.6 weights (including the extracted vision_tower) © OpenBMB / THUNLP (Tsinghua University) (2024-2026). SigLIP2 foundational vision-encoder design © Google LLC (2024-2026). NOESIS vision-only extraction, BF16 SiglipVisionModel rebuild, integration metadata, sealed-rule wiring: © AMAImedia (NOESIS DHCF-FNO project) 2026 (redistributed under Apache 2.0).

Commercial use is permitted subject to the standard Apache 2.0 preservation requirements (copyright + LICENSE + NOTICE-equivalent attribution must travel with redistributions). See LICENSE in this folder for the full Apache 2.0 text plus the NOESIS attribution banner and required-attribution block.

The upstream MiniCPM-V authors' disclaimer about MLLM content generation (see LICENSE § Statement) is inherited verbatim by this derivative.

Bundle identity

• Parent bundle: NOESIS-3.5B-A0.5B-DUBBING-FILM
• Vendored component: NOESIS-MiniCPM-V-4.6-SigLIP2-BF16 (SigLIP2 vision tower, BF16, 996 MB)
• Original extraction date: 2026-05-17
• Source card last updated: 2026-05-27

---

📚 Theorem reference layout (post 2026-05-20 split). This document references NOESIS Theorem entries (NT-N / R-X / B-N / Decision-N). The project-level theorem files and split tooling are not bundled in this model-only repository; the original path names are retained as plain code references for provenance without creating dead Hugging Face links.

Project references: docs/NOESIS_04_THEOREM.md, docs/theorem/00_INDEX.md, docs/theorem/ANCHOR_MAP.json, scripts/split_theorem.py, and scripts/migrate_theorem_refs.py.