Instructions to use ngqtrung/video-8b-grpo-ppexplore with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ngqtrung/video-8b-grpo-ppexplore with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ngqtrung/video-8b-grpo-ppexplore")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ngqtrung/video-8b-grpo-ppexplore")
model = AutoModelForMultimodalLM.from_pretrained("ngqtrung/video-8b-grpo-ppexplore")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ngqtrung/video-8b-grpo-ppexplore with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ngqtrung/video-8b-grpo-ppexplore"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ngqtrung/video-8b-grpo-ppexplore",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ngqtrung/video-8b-grpo-ppexplore

SGLang

How to use ngqtrung/video-8b-grpo-ppexplore with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ngqtrung/video-8b-grpo-ppexplore" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ngqtrung/video-8b-grpo-ppexplore",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ngqtrung/video-8b-grpo-ppexplore" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ngqtrung/video-8b-grpo-ppexplore",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ngqtrung/video-8b-grpo-ppexplore with Docker Model Runner:
```
docker model run hf.co/ngqtrung/video-8b-grpo-ppexplore
```

Qwen3-VL-8B · Video-MC · GRPO + ppexplore (τ0.95, cold-start)

RLVR post-training of Qwen/Qwen3-VL-8B-Instruct on multiple-choice video QA with fully-async GRPO and an entropy-aware token-dropout exploration ("ppexplore", τ=0.95) at the rollout stage, cold-started from the stock 8B-Instruct. This is experiment_name=grpo_video_4node_full_v3_24f100k_8b_base_ppexplore_v1, global_step_140 (the keeper).

On video, exploration is a WASH: cold-start ppexplore (τ0.95) peaks at offline full-set val mean_accuracy 0.4890 @ step 140 — statistically tied with the cold-start base (0.4918) and SFT-770 warmstart (0.4845). The token-dropout exploration that won +4.6 pt on OMR delivers no gain on video; this run sits in the same ~0.485 dead-heat band.

Results

Offline full-set eval (VideoMME-v1 2700 + PerceptionComp 1108 + Video-Holmes 1837 = 5645 rows), scored with the repo's vero compute_score. mean = macro-mean of the 3 bench accuracies.

Keeper = global_step_140 (peak):

metric	mean	videomme	holmes	perceptioncomp
ppexplore τ0.95 @140 (keeper)	0.4890	0.656	0.465	0.346
cold-base RL keeper @80 (sibling)	0.4918	0.658	0.474	0.343
SFT-770 RL keeper @180 (sibling)	0.4845	0.657	0.451	0.345
stock Qwen3-VL-8B ckpt-0 (zero-shot)	0.4444	0.6426	0.4143	0.2762

Trajectory (mean): 0.451 @20 → 0.470 @40 → 0.477 @60 → 0.488 @100 → 0.489 @140 → faded late (0.477 @200, 0.480 @260). Ran to ~step 780 (near 2 epochs); further training brought no val gain — same post-peak fade as every other video run. ppexplore's lower train reward did not translate to higher val.

Training

Base model: stock Qwen/Qwen3-VL-8B-Instruct (cold-start, resume_mode=auto).
Framework: fork of volcengine/verl — ngquangtrung57/verl@videorl-mods. Fully-async GRPO: FSDP2 trainer + vLLM rollouter.
Warm start: none (cold-start base).
Reward: dapo-style score = 0.8·accuracy + 0.2·format (FORMAT_WEIGHT=0.2, FORMAT_MIN_THINK_CHARS=100). No KL penalty.
Topology: 4-node 2+2 — 2 trainer nodes (16-GPU FSDP2, dp=16) + 2 rollout nodes (16 GPU, vLLM TP=2 → 8 replicas). H100×8 per node.
Batch: ppo_mini_batch_size=16 × require_batches=4 × rollout.n=8 = 512 trajectories/step.
Data: GROUP_VIDEO_TRAIN_MC_24F100K (5 video-MC parquets, 24 frames / 100k pixels).
Optim / seq: lr 1e-6, warmup 25 steps; total_epochs=2; clip_ratio 0.2 / 0.3 (clip_c=10.0); max_prompt_length=17408, max_response_length=16384; enforce_eager=true; gpu_memory_utilization=0.75; staleness 0.4.
Exploration block (entropy-aware token-dropout at rollout; byte-identical to the OMR-winner τ0.95 block):

key	value
`enable`	true
`trigger_mode`	high
`top_prob_threshold` (τ)	0.95
`k_explore`	4 (of n=8 rollouts explore; 4 stay clean)
`prompt_exploration_prob`	0.5
`deterministic`	true
`perturb_prob`	1.0
`mask_from_loss`	true
`drop_top_k`	1
`restrict_to_think_region`	true
`selection_seed`	42

Validation: inline val OFF (test_freq=10000); video val is offline full-set eval on a dedicated 8×H100 node (vLLM TP1), every 20 fit-steps.
Train metrics: ~~219 s/step; final reward 0.858; final response_length ~164 tok; 780 steps trained (~~2 epochs).

W&B

Project verl_fully_async (entity quangtrung5705-nanyang-technological-university-singapore). Train metrics only — video val is offline, not on W&B: https://wandb.ai/quangtrung5705-nanyang-technological-university-singapore/verl_fully_async/runs/88lmybpk

Intended use / limitations

Research checkpoint — the cold-start exploration arm of an 8B video study. Headline finding: on video, exploration is a documented dead-heat — the OMR-winning recipe applied to video does not break the ~0.485 full-set val ceiling (this run, base, SFT-warmstart, and several further exploration ablations all land ~0.485-0.49). Useful as an A/B reference, not as a "better video model" — prefer video-8b-grpo-base (0.4918) if you just want the best keeper. Multiple-choice video QA, <think>…</think> then-answer format. No safety/RLHF alignment beyond the base.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "ngqtrung/video-8b-grpo-ppexplore"
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)

messages = [{
    "role": "user",
    "content": [
        {"type": "video", "video": "clip.mp4"},
        {"type": "text", "text": "Answer the multiple-choice question. Reason inside <think>...</think>, then give the final letter."},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt"
).to(model.device)
out = model.generate(**inputs, max_new_tokens=1024)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

Citation / lineage

Base model: Qwen3-VL-8B-Instruct (Qwen team). Inherits the Qwen3-VL license — review the base model's terms; the Apache-2.0 tag refers to this repo's RLVR training artifacts.
Framework: verl (volcengine/verl), fork ngquangtrung57/verl@videorl-mods; fully-async GRPO (FSDP2 + vLLM).
Method: entropy-aware token-dropout exploration ("ppexplore", τ=0.95) at the rollout stage. Part of a controlled OMR/Video exploration study on Qwen3-VL-8B (docs/experiments_summary_8b.md).

Downloads last month: 18

Safetensors

Model size

9B params

Tensor type

BF16

Model tree for ngqtrung/video-8b-grpo-ppexplore

Base model

Qwen/Qwen3-VL-8B-Instruct

Finetuned

(333)

this model

Collection including ngqtrung/video-8b-grpo-ppexplore

Qwen3-VL-8B RLVR — Models (v1)

Collection

Qwen3-VL-8B GRPO RLVR checkpoints from a token-dropout exploration study. OMR ppexplore=winner (0.714); video ~0.485 dead-heat. • 6 items • Updated 5 days ago