Instructions to use yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen

SGLang

How to use yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen with Docker Model Runner:
```
docker model run hf.co/yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen
```

[📖 arXiv Paper] [📊 MM-RLHF Data] [📝 Homepage] [🏆 Reward Model]

[🔮 MM-RewardBench] [🔮 MM-SafetyBench] [📈 Evaluation Suite] [📊 Training Code]

The Next Step Forward in Multimodal LLM Alignment

[2025/02/10] 🔥 We are proud to open-source MM-RLHF, a comprehensive project for aligning Multimodal Large Language Models (MLLMs) with human preferences. This release includes:

A high-quality MLLM alignment dataset.
A strong Critique-Based MLLM reward model and its training algorithm.
A novel alignment algorithm MM-DPO.
Two new benchmarks.

Our dataset and algorithms enable consistent performance improvements across 10 dimensions and 27 benchmarks.\n

Use

Intended use

The model was trained on MM-RLHF data and have the ability to interact with images, multi-image and videos.

Feel free to share your generations in the Community tab!

Generation

We provide the simple generation process for using our model. For more details, you could refer to Github.

Citation

If you find it useful for your research and applications, please cite related papers/blogs using this BibTeX:

@article{zhang2025mm,
  title={MM-RLHF: The Next Step Forward in Multimodal LLM Alignment},
  author={Zhang, Yi-Fan and Yu, Tao and Tian, Haochen and Fu, Chaoyou and Li, Peiyan and Zeng, Jianshu and Xie, Wulin and Shi, Yang and Zhang, Huanyu and Wu, Junkang and others},
  journal={arXiv preprint arXiv:2502.10391},
  year={2025}
}

Downloads last month: 47

Safetensors

Model size

8B params

Tensor type

BF16

Collection including yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen

MM-RLHF

Collection

The Next Step Forward in Multimodal LLM Alignment • 4 items • Updated Feb 17, 2025

Paper for yifanzhang114/MM-RLHF-Reward-7B-llava-ov-qwen

MM-RLHF: The Next Step Forward in Multimodal LLM Alignment

Paper • 2502.10391 • Published Feb 14, 2025 • 34