qgfvadfuvads/Q-Prefer-D2

Q-Prefer D2 Video Reward Model

Q-Prefer D2 is a Qwen3-VL-4B-based reward model for T2V and I2V videos. This repository contains a PEFT adapter and the tokenizer files needed by the qprefer_reward inference package. Install the accompanying GitHub repository before loading this artifact; this is not a standalone Transformers AutoModel checkpoint.

Source code and installation are provided by the accompanying Q-Prefer GitHub repository.

Architecture and supported outputs

• Base: Qwen/Qwen3-VL-4B-Instruct
• Base revision: ebb281ec70b05090aa6165b016eac8ec08e71b17
• LoRA: rank 64, alpha 128, dropout 0.05
• Input: eight uniformly sampled video frames and an optional I2V reference image
• Head: Linear(2560, 3, bias=False) evaluated at three special reward tokens
• Supported outputs: Visual Quality (VQ) and Text Alignment (TA)

The checkpoint retains a Motion Quality (MQ) output for compatibility, but D2 did not supervise MQ. Do not use it as a downstream reward.

The original D2 run did not optimize or save the three newly added input-token embedding rows. This release freezes the compatibility rows used by the fully validated inference artifact. They are required for portable inference, cannot be recovered from the old adapter alone, and are not claimed to be training-time exported tensors.

Usage

import torch
from qprefer_reward import QPreferConfig, QPreferScorer

scorer = QPreferScorer(
    QPreferConfig(
        adapter="qgfvadfuvads/Q-Prefer-D2",
        adapter_revision="IMMUTABLE_HF_COMMIT",
        device="cuda",
        dtype=torch.bfloat16,
    )
)

scores = scorer.score_batch(
    videos=["generated.mp4"],
    prompts=["A red fox runs through a snowy forest."],
)
print(scores.visual_quality, scores.text_alignment)

For I2V, also pass reference_images=["condition.png"]. The full architecture, tensor rollout API, GRPO adapter, installation instructions, and limitations are documented in the accompanying GitHub repository.

Integrity and limitations

Validated adapter SHA256:

3f3788b942cb8e259582ee6cb2ac7c7bb2592a63e7183fe89c8434aef86476cf

Validated special-token embedding SHA256:

52c5aa200d63d093e9d46b16cfbcca14f1b00e0baf4fb5708de414d01511007f

D2 was trained from Gemini-generated preference labels. A higher Q-Prefer reward is an optimization diagnostic, not independent evidence of human preference. Downstream claims require held-out blind human evaluation and independent metrics.

---

中文说明

Q-Prefer D2 是基于 Qwen3-VL-4B 的 T2V/I2V 视频奖励模型。本模型仓库保存 PEFT adapter 和 qprefer_reward 推理包需要的 tokenizer 文件。使用前必须先 安装配套 GitHub 代码；这里不是能够单独通过 Transformers AutoModel 加载的 完整 checkpoint。

模型使用固定 Qwen base revision、rank 64 LoRA、8 帧均匀采样，以及一个 Linear(2560, 3, bias=False) reward head。对外只支持 Visual Quality（VQ）和 Text Alignment（TA）。checkpoint 中的 MQ 仅用于结构兼容，D2 没有监督 MQ， 因此不能把它作为下游 reward。

原始 D2 训练没有优化或保存三个新增输入 token 的 embedding。本发布包固定了 经过完整验证的推理 artifact 所使用的 compatibility rows。仅靠旧 adapter 无法 恢复这些行；它们用于保证可移植推理，也不是“训练时导出的 tensor”。

T2V/I2V 文件、内存 rollout tensor、GRPO 接入方法和完整局限性请查看配套 GitHub 仓库。D2 使用 Gemini preference labels 训练，因此 reward 上升不能 替代独立盲测人评和独立自动指标。