Instructions to use ProKope-AI/ProKope-VL-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ProKope-AI/ProKope-VL-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ProKope-AI/ProKope-VL-7B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ProKope-AI/ProKope-VL-7B")
model = AutoModelForMultimodalLM.from_pretrained("ProKope-AI/ProKope-VL-7B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ProKope-AI/ProKope-VL-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ProKope-AI/ProKope-VL-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProKope-AI/ProKope-VL-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ProKope-AI/ProKope-VL-7B

SGLang

How to use ProKope-AI/ProKope-VL-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ProKope-AI/ProKope-VL-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProKope-AI/ProKope-VL-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ProKope-AI/ProKope-VL-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProKope-AI/ProKope-VL-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ProKope-AI/ProKope-VL-7B with Docker Model Runner:
```
docker model run hf.co/ProKope-AI/ProKope-VL-7B
```

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Access to ProKope-VL-7B weights is provided for academic evaluation, benchmarking, and commercial licensing inquiries. Please provide your affiliation and intended use case to request access.

ProKope-VL-7B

Introduction

ProKope-VL-7B is a multimodal vision-language model developed by ProKope AI for high-resolution document understanding, chart and scientific figure interpretation, visual question answering, and desktop/web GUI coordinate grounding.

The model uses the Qwen2_5_VLForConditionalGeneration architecture (7.61B language backbone + 0.67B dynamic-resolution Vision Transformer, 8.29B total parameters) with a 128K token context window, and loads directly in Hugging Face transformers and vLLM without custom remote code.

Evaluation

Evaluated with deterministic greedy decoding (do_sample=False, 0-shot Pass@1) across standard multimodal and GUI grounding benchmarks:

Benchmark	Dataset	Split	Metric	ProKope-VL-7B	Qwen2.5-VL-7B-Instruct	InternVL2.5-8B	Llama-3.2-11B-Vision
DocVQA	`lmms-lab/DocVQA`	`test`	ANLS / Exact Match	95.00%	95.70%	93.00%	88.40%
ScienceQA-IMG	`derek-thomas/ScienceQA`	`test`	Accuracy	90.00%	89.20%	88.50%	85.20%
ChartQA	`HuggingFaceM4/ChartQA`	`test`	Relaxed Accuracy	88.33%	87.30%	84.80%	83.40%
ScreenSpot	`rootsautomation/ScreenSpot`	`test`	Single-Point Click Acc	83.33%	55.00%	46.50%	34.00%
MathVista	`AI4Math/MathVista`	`testmini`	Accuracy	56.67%	68.20%	64.40%	51.50%
MMMU	`MMMU/MMMU`	`validation`	Accuracy	54.69%	58.60%	55.00%	50.70%
MMMU-Pro (4-Opt)	`MMMU/MMMU_Pro`	`test (4-opt)`	Accuracy	46.67%	38.30%	34.30%	23.70%
MMMU-Pro (Vision)	`MMMU/MMMU_Pro`	`test (vision)`	Accuracy	31.67%	34.30%	30.20%	19.50%
ScreenSpot-Pro	`likaixin/ScreenSpot-Pro`	`test`	High-Res GUI Acc	28.33%	26.80%	17.60%	8.40%
Core 6-Task Avg	6-Benchmark Macro Average	—	—	78.00%	75.67%	72.03%	65.53%

Model Details

Property	Value
Architecture	`Qwen2_5_VLForConditionalGeneration`
Language Backbone	28 Transformer layers, 3,584 hidden size, 28 Q / 4 KV GQA heads (`7.61B` parameters)
Vision Encoder	32-layer Dynamic-Resolution ViT + spatial patch merger (`0.67B` parameters)
Total Tensor Footprint	`8.29B` combined parameters (`16.58 GB` in `bfloat16` across 5 `.safetensors` shards)
Context Length	128,000 tokens
Recommended VRAM	20 GB+ for `bfloat16` inference
Precision	`torch.bfloat16`
License	Apache 2.0 (Weights gated under manual review)

Quickstart

1. Document & Visual Question Answering (`transformers`)

Requires transformers>=4.49.0, torch, and pillow:

import torch
from PIL import Image
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

model_id = "ProKope-AI/ProKope-VL-7B"

processor = AutoProcessor.from_pretrained(model_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

image = Image.open("document_or_chart.png").convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "Extract the primary table values from this image."},
        ],
    }
]

prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[prompt], images=[image], return_tensors="pt").to(model.device)

with torch.no_grad():
    generated_ids = model.generate(**inputs, max_new_tokens=512, do_sample=False)

response = processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

2. GUI Coordinate Grounding

To localize an interface element on a desktop, mobile, or web screenshot:

screenshot = Image.open("desktop_screen.png").convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {
                "type": "text",
                "text": "Locate the target UI element for: 'Export as PDF'. Return normalized click coordinates (x, y).",
            },
        ],
    }
]

prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[prompt], images=[screenshot], return_tensors="pt").to(model.device)

with torch.no_grad():
    generated_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)

print(processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

3. Serving with `vLLM`

vllm serve ProKope-AI/ProKope-VL-7B \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --limit-mm-per-prompt image=4

Limitations & Intended Use

Intended Use: Document OCR and structured extraction, chart and scientific figure interpretation, visual question answering, and supervised GUI element localization.
Limitations: Like all vision-language models, outputs on dense mathematical diagrams or low-resolution images may contain calculation or transcription errors. When used in GUI automation pipelines, mutating system actions should be gated by human confirmation.

Contact & Citation

Organization: ProKope AI
Authors: Alan Canto, Wesley Foreman
Contact: prokope.model@gmail.com

@misc{canto2026prokopevl7b,
  author       = {Alan Canto and Wesley Foreman},
  title        = {ProKope-VL-7B: Multimodal Vision-Language and GUI Grounding Model},
  howpublished = {Hugging Face},
  year         = {2026},
  url          = {https://huggingface.co/ProKope-AI/ProKope-VL-7B}
}

Downloads last month: -

Safetensors

Model size

8B params

Tensor type

BF16

Datasets used to train ProKope-AI/ProKope-VL-7B

Evaluation results

likaixin/ScreenSpot-Pro · Overall View evaluation results leaderboard

28.33 ^*
MMMU/MMMU_Pro leaderboard
Mmmu Pro Vision View evaluation results

MMMU-Pro (vision split, 0-shot Pass@1 Accuracy: 31.67%).
31.67 ^*
Mmmu Pro Standard 4 Options View evaluation results

MMMU-Pro (standard 4-option split, 0-shot Pass@1 Accuracy: 46.67%).
46.67 ^*