PixelBench-Molmo2-4B-PRT-s1

Molmo2-4B PRT, seed s1, from the paper's four-seed comparison. Instruction-tuned on 500,000 LLaVA-OneVision-1.5 instruction examples and 2,000 Pixel trajectories at 16×16 and 32×32, starting from the Molmo2-4B-Pretrain step-32,000 checkpoint. All model parameters were trained.

Part of Can LLMs Understand and Generate Images Pixel by Pixel? Project · Code and evaluation

Usage

Install a CUDA-compatible PyTorch build, then:

pip install transformers==4.57.6 accelerate pillow torchvision

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "haoquan03/PixelBench-Molmo2-4B-PRT-s1"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype="auto", device_map="auto", attn_implementation="sdpa", trust_remote_code=True
).eval()

messages = [{"role": "user", "content": [
    {"type": "image", "image": Image.open("image.jpg").convert("RGB")},
    {"type": "text", "text": "Describe this image."},
]}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
    output = model.generate(**inputs, max_new_tokens=256, do_sample=False, use_cache=True)
print(processor.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

Training settings and the evaluation protocol are available in the code repository.

Citation

@article{zhang2026pixelbench,
    title={Can {LLMs} Understand and Generate Images Pixel by Pixel?},
    author={Haoquan Zhang and Feiyang Liu and Juanxi Tian and Wanhua Li and Haodong Duan and Weiyang Liu},
    journal={arXiv preprint arXiv:TBA},
    year={2026},
    url={https://haoquanzhang.github.io/pixelbench/}
}