Instructions to use PixelProof/gemma-4-31b-pixelproof-lora-adapters with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries
PEFT
How to use PixelProof/gemma-4-31b-pixelproof-lora-adapters with PEFT:
```
Task type is invalid.
```

How to use PixelProof/gemma-4-31b-pixelproof-lora-adapters with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="PixelProof/gemma-4-31b-pixelproof-lora-adapters")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("PixelProof/gemma-4-31b-pixelproof-lora-adapters", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PixelProof/gemma-4-31b-pixelproof-lora-adapters with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PixelProof/gemma-4-31b-pixelproof-lora-adapters"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PixelProof/gemma-4-31b-pixelproof-lora-adapters",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/PixelProof/gemma-4-31b-pixelproof-lora-adapters

SGLang

How to use PixelProof/gemma-4-31b-pixelproof-lora-adapters with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PixelProof/gemma-4-31b-pixelproof-lora-adapters" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PixelProof/gemma-4-31b-pixelproof-lora-adapters",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PixelProof/gemma-4-31b-pixelproof-lora-adapters" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PixelProof/gemma-4-31b-pixelproof-lora-adapters",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use PixelProof/gemma-4-31b-pixelproof-lora-adapters with Docker Model Runner:
```
docker model run hf.co/PixelProof/gemma-4-31b-pixelproof-lora-adapters
```

PixelProof LoRA adapters for Gemma 4 31B

LoRA adapters for google/gemma-4-31B-it trained with supervised fine-tuning (SFT) on visual questions generated by PixelProof, as described in the paper PixelProof: Visual Question Generation through Forward–Inverse Agreement. In PixelProof, a coding agent writes a forward program that draws images and computes their answers and an inverse program that must recover each answer from the image alone; a question is kept only if the two agree.

Each adapter is stored in its own subfolder. The repository contains no merged weights.

Training

Training uses transformers.Trainer and PEFT LoRA adapters on the language model's attention and feed-forward projections; the vision tower is not directly adapted. The main recipe uses rank 16, scaling (alpha) 32, dropout 0.05, bfloat16, one epoch, learning rate 1e-4, cosine decay, 3% warm-up, batch size two, and eight gradient-accumulation steps. The loss covers only the assistant answer and the turn-end token. Adapter names ending in -accum16 use sixteen accumulation steps.

The main training data contains only profile-steered worlds. The fixed split assigns 748 of 793 replay-verified worlds to training (27,762 training instances, with 555 reserved for validation) and holds out 45 worlds, one per model–profile campaign. Worlds, not instances, are held out, because instances from the same world share a sampler and renderer.

Adapters and paper results

Run	Subfolders	Paper result
Main recipe: the fixed split and three further random splits (`-variant-seed101` to `-variant-seed103`)	`gemma4-31b-lora-bf16-profile-steered`, `gemma4-31b-lora-bf16-profile-steered-variant-seed101`, `gemma4-31b-lora-bf16-profile-steered-variant-seed102`, `gemma4-31b-lora-bf16-profile-steered-variant-seed103`	Held-out profile-steered worlds; transfer to model-feedback-steered worlds; 16 external benchmarks
Answer-option order control: fixed split with every answer reassigned to a balanced-random position	`gemma4-31b-lora-bf16-profile-steered-optshuffle`	Answer-option order analysis
Leave-one-profile-out, one adapter per excluded discovery profile	`gemma4-31b-lora-bf16-profile-steered-holdout-<profile>` (9 adapters)	Transfer to excluded profiles
Direct training on model-feedback-steered worlds (eight-step accumulation)	`gemma4-31b-lora-bf16-model-feedback`	Direct training on model-feedback-steered questions

<profile> is one of the nine discovery profiles: correspondence, measurement, prior_conflict, reveal_declared_transform, reveal_global_structure, search, state_tracking, topology, tracing.

Results for Gemma 4 31B reported in the paper:

Held-out profile-steered worlds, mean of four splits: 64.4% → 74.6% (+10.2 pp). The average over the three fine-tuned models is +10.7 pp.
Excluded discovery profiles, mean over nine profiles: +7.3 pp (average over the three models: +8.5 pp).
Transfer to 1,260 instances from 252 model-feedback-steered worlds: 30.7% → 37.8% (+7.1 pp; average over the three models: +5.3 pp).
16 external benchmarks, mean over four splits: 60.0 → 61.1 (+1.0 points; average over the three models: +1.9 points).
Direct training on model-feedback-steered worlds, 250 instances from 50 held-out worlds: +19.2 pp.

Additional adapters not reported in the paper

The following adapters come from additional runs that the paper does not report. They are released for completeness only.

gemma4-31b-lora-bf16
gemma4-31b-lora-bf16-model-feedback-variant-seed101
gemma4-31b-lora-bf16-model-feedback-variant-seed102
gemma4-31b-lora-bf16-model-feedback-variant-seed103
gemma4-31b-lora-bf16-profile-steered-4source
gemma4-31b-lora-bf16-profile-steered-variant-seed101-balanced
gemma4-31b-lora-bf16-profile-steered-variant-seed102-balanced
gemma4-31b-lora-bf16-profile-steered-variant-seed103-balanced
gemma4-31b-lora-bf16-spatially-steered
gemma4-31b-lora-bf16-spatially-steered-variant-seed101
gemma4-31b-lora-bf16-spatially-steered-variant-seed102
gemma4-31b-lora-bf16-spatially-steered-variant-seed103

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

base_id = "google/gemma-4-31B-it"
repo_id = "PixelProof/gemma-4-31b-pixelproof-lora-adapters"
subfolder = "gemma4-31b-lora-bf16-profile-steered"

processor = AutoProcessor.from_pretrained(repo_id, subfolder=subfolder)
model = AutoModelForImageTextToText.from_pretrained(base_id, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, repo_id, subfolder=subfolder)

Prompt the model with only the image, the question text, and the answer options. The evaluation data are available in PixelProof/PixelProof-Evaluation and PixelProof/PixelProof-Difficulty-Feedback.

Limitations

All PixelProof training data are synthetic two-dimensional images, and the gains may not extend to natural images. On the external benchmarks, some benchmarks show smaller gains or regressions. Agreement between the forward and inverse programs shows that an answer can be recovered from the image, but not necessarily that the question is clearly worded.

Project resources

Downloads last month: -

Model tree for PixelProof/gemma-4-31b-pixelproof-lora-adapters

Base model

google/gemma-4-31B

Finetuned

google/gemma-4-31B-it

Adapter

(308)

this model

Collection including PixelProof/gemma-4-31b-pixelproof-lora-adapters

Models and Datasets

Collection

Model adapters and evaluation and difficulty-feedback datasets for PixelProof: Visual Question Generation through Forward–Inverse Agreement. • 5 items • Updated 1 day ago