gatilin/glm5_3_flash_vision

GLM-5.3-Flash-BF16 — Standalone Native Vision Bundle

Visual weights extracted directly from zai-org/GLM-5.3-Flash-BF16, fixed to revision f9fed3bbc82291073373d67452e9f25317ef7577. Includes the complete native visual module: patch embedding, 24 vision blocks and the trained merger/projection to 4096 dimensions.

Only visual/connector weights are included. No language-model weights, language-model head, or additional training. This bundle uses the original Transformers visual classes through the included explicit loader; it is not a timm remap and does not register an AutoModel remote-code implementation.

Contents and boundaries

The complete native visual module includes its trained merger. Backbone tokens and projected features are different output spaces. Do not treat this checkpoint as an image classification model. In GLM, last_hidden_state is already spatially downsampled and projected to the LLM width; it is not the raw 1024-dimensional block output. The upstream video path flattens temporal grids before calling the visual module; see the validation script.

• model.safetensors: 347 tensors, 563,627,008 parameters across the selected visual modules, stored as BF16; 1.127 GB.
• source_config.json: complete source configuration for correctly constructing visual components; its text configuration is metadata only.
• vision_config.json: source vision subconfiguration.
• config.json: bundle provenance and loader metadata, not a full VLM configuration.
• tensor_mapping.json: exact source keys, original shard offsets, and output keys.
• vision_model.py: local strict loader; constructs visual modules only.
• extract_multimodal_vision.py, validate_multimodal_vision.py: reproducible extraction and verification scripts.
• extraction.json, validation.json: file checksum, provenance and test results.
• Source preprocessing configuration files where supplied, plus SOURCE_README.md and license/attribution files.

Download and load

pip install -r requirements.txt

from huggingface_hub import snapshot_download
import sys
root = snapshot_download('gatilin/glm5_3_flash_vision')
sys.path.insert(0, root)

import torch
from vision_model import load_model
model = load_model(root)
# Synthetic native flattened temporal/spatial patches, not raw BCHW pixels.
pixel_values = torch.randn(16, 3 * 2 * 14 * 14)
grid_thw = torch.tensor([[1, 4, 4]])
with torch.inference_mode():
    result = model(pixel_values, grid_thw=grid_thw)
print(result.last_hidden_state.shape)
print(result.pooler_output.shape)       # [4, 4096]

The example uses synthetic, already-packed visual input tensors to show the exact native API. For real images/videos, use the upstream processor at the pinned source revision and preserve its resize, normalization, patch order, temporal padding, and grid metadata. Raw BCHW images cannot be passed directly to the flattened/packed API. This repository does not include a tokenizer or text-generation pipeline. load_model(..., dtype=torch.bfloat16, device='cuda') is available for compatible hardware; FP32 CPU is the tested validation setting.

Verification

Every exported tensor's shape, dtype and raw bytes were compared with the downloaded exact source byte ranges: all identical. Only source key prefixes were removed; no reshaping, quantization, retraining, or numerical conversion was applied to stored weights.

Both strict state loading and 4 native-feature comparisons passed. Reference outputs came from the original Transformers parent's get_image_features / applicable get_video_features methods, with weights reconstructed directly from source ranges; unused language-model modules were not allocated. Tests covered square and rectangular patch grids, packed multiple-image inputs, and two temporal groups for video. Maximum absolute errors: hidden features 0, projected features 0. Tolerance: rtol=1e-5, atol=1e-5.

A native-input 448×448 resolution smoke test also passed, producing finite outputs. Tests used torch 2.14.1, Transformers 5.20.1, FP32 CPU, SDPA, seed 17. These are extraction and visual API consistency tests, not end-to-end real-image processing, language generation, task benchmarks, or GPU/BF16 accuracy claims.

Reproduce

python extract_multimodal_vision.py glm5_3_flash --output ./reproduced --refresh-source
python validate_multimodal_vision.py ./reproduced/glm5_3_flash

The extractor uses Python's standard library and validated HTTP ranges to download only selected visual tensors. --refresh-source refills the resumable .download ranges for source-byte revalidation. The validation script imports vision_model.py from the same directory as itself. The published bundle excludes download caches.

中文说明

本仓库保留原生视觉塔及其训练过的压缩/投影模块，通过随附的 vision_model.py 加载，不会构造语言模型。已完成逐张量源字节核验、严格加载、原生视觉特征与投影输出对比，以及 448×448 分辨率前向检查。主权重保持 BF16。输入需要使用上游处理器生成的原生 patch 排列和网格信息；这里的示例及验证覆盖原生视觉张量接口，不代表整套实图到文本生成或下游精度测试。

License and source

Follow the upstream model's mit license and attribution. Original model documentation is preserved in SOURCE_README.md. This is an independent extraction without affiliation with or endorsement by the original authors. The included loader depends on the upstream Transformers implementation and its license.