VectraYX-Vision-1B

A sub-2B vision–language model specialized for cybersecurity imagery — disassembly/decompiler panes (IDA, Ghidra), packet captures (Wireshark), scanner and post-exploitation terminals (Nmap, Metasploit), and memory-forensics dumps (Volatility) — that answers in Spanish, emits structured visual reasoning via native <|think|> tokens before its answer, invokes external tools through the Model Context Protocol using native <|tool_call|> tokens, and exports cleanly to llama.cpp's LLaVA mmproj format for fully on-premise, air-gapped deployment.

This is the GGUF export (model + mmproj) for offline inference. For the text-only backbone see jsantillana/vectrayx-1b.

Architecture

Component	Value
Visual encoder	SigLIP-so400m-patch14-384 (frozen, 400M, 1152-d, 729 patches)
Projector	MLP: 1152→4096→2048, GELU + LayerNorm (~13.1M params)
Language backbone	VectraYX-1B: 22 layers, 2048 d_model, 5504 d_ffn, GQA 16/4, SwiGLU
Backbone positional encoding	RoPE (θ=1e6) on 3/4 layers, NoPE on every 4th layer
Vocabulary	32,768 (BPE)
Special tokens	`<
Total parameters	~1.04B (backbone) + 400M (encoder) + 13M (projector)

Files

• model.gguf (2.22 GB, F16) — the LLM decoder (also usable standalone for text as in vectrayx-1b)
• mmproj.gguf (823 MB, F16) — the SigLIP encoder + MLP projector (the vision tower)

Both are needed for multimodal inference; load both in llama.cpp / Ollama.

Training

Four-phase curriculum (all phases complete):

1. Phase 1 — 9.2B tokens general Spanish pretraining (backbone)
2. Phase 2 — ≈50B tokens three-block curriculum (backbone)
3. Phase 3 — ≈6B tokens tooling SFT establishing native <|tool_call|> + <|think|> (backbone)
4. Vision phase — 4a (projector alignment, ~228 steps), 4b (instruct, projector+backbone), 4c (think+tools). This checkpoint is the SFT-v2 continuation at step 400, warm-started from the v3B run (step 1900 of a low-LR phase-4b).

Total vision-phase compute: ≈2.2h of 2×A100-40GB wall time; ≈16M tokens.

Honest status

B6/B7 (vision benchmark) scores are near zero at the current training volume (400–1900 vision steps). The model generates fluent Spanish but ignores image content — visual grounding at this scale requires more SFT steps or a denser annotated screenshot corpus. This is a template/infrastructure release, not a quality release. Baseline context:

Model	B6 tool-id	B6 correct	B7 chain-quality
GPT-4o (frontier ceiling)	0.940	0.000	0.940
Phi-4-multimodal (4.5B)	0.440	0.000	0.220
VectraYX-Vision-1B (v3B, step 1900)	0.080	0.060	0.025
VectraYX-Vision-1B (SFT-v2, step 400) — this checkpoint	0.020	0.000	0.010
1B backbone, no image (control)	0.020	0.000	0.000

The text-only control confirms the benchmark is not trivially answerable from text alone — the with-image v3B row (0.08) exceeds the no-image control (0.02).

Usage (llama.cpp multimodal)

# Load model + vision tower
./llama-cli -m model.gguf --mmproj mmproj.gguf \
  --image /path/to/screenshot.png \
  -p "¿Qué herramienta de seguridad muestra esta captura y qué hace?" \
  --temp 0.7 -n 512

Usage (Ollama)

# Register the model with its mmproj
ollama create vectrayx-vision-1b \
  -f <(printf 'FROM ./model.gguf\nPARAMETER temperature 0.7\nPARAMETER num_predict 512\n')
# (mmproj must be co-located for multimodal; see Ollama LLaVA docs)
ollama run vectrayx-vision-1b

Notes / known limitations

• Visual grounding is weak. The model is primarily an architecture/data/curriculum release pending a longer SFT run. Do not expect useful visual QA yet.
• NoPE export: NoPE layers export as RoPE layers with rotary application disabled per layer. The GGUF RoPE permute fix uses stack([w1,w2], dim=2) interleaved.
• Export is GGUF-splittable: the decoder checkpoint contains no visual weights and vice versa, matching the LLaVA model.gguf / mmproj.gguf convention.
• This is the first released model to combine periodic NoPE layers with an injected visual-token block; the NoPE×vision interaction is an open research question (see the accompanying paper for the ablation design).

Citation

See the accompanying paper, VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use.

Author

Juan S. Santillana — DevOps engineer (Globant). Institutional affiliation approval pending. Weights, benchmarks, and configs are released; the tool-calling MCP Modelfile will follow the visual-prompt-injection and over-triggering safety evaluation.