InserloftResearch/Valdam-Vision-2.5

👁️ Valdam Vision 2.5

Valdam Vision 2.5 is a Multimodal Vision-Language Model (VLM) designed for scene description, spatial OCR, and Zero-Shot content moderation.

This release integrates visual patch projection from a pretrained Vision Transformer with a modular autoregressive Transformer Decoder.

---

🌟 Key Features

• Autoregressive Text Generation: Generates descriptions by applying cross-attention over visual patch embeddings.
• Zero-Shot Moderation (Valdam Shield): Content screening powered by CLIP ViT-L/14 semantic similarity matching.
• Spatial Layout OCR: Reconstructs read order and paragraphs from designs using spatial coordinates.
• Inference Latency: Optimized pipeline with an average response time of ~0.08s on standard hardware.

---

📦 Repository Structure

• model.safetensors: Model weights in safe-tensors format containing header metadata.
• architecture.py: Model architecture definition (ViT Encoder & Transformer Decoder).
• inference.py: Script to initialize and run local predictions.
• config.json: Hyperparameters and model configuration.
• vocab.json: Mapping of vocabulary tokens.

---

🚀 Quickstart

from inference import ValdamVisionLocalPredictor

# Initialize the predictor
predictor = ValdamVisionLocalPredictor(
    safetensors_weights_path="model.safetensors",
    config_path="config.json",
    vocab_path="vocab.json",
    device="cuda"
)

# Run prediction on a preprocessed image tensor
result = predictor.predict(image_tensor)
print(f"Result: {result}")

Website

Visit us!