Instructions to use FINAL-Bench/Darwin-180B-RSI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FINAL-Bench/Darwin-180B-RSI with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="FINAL-Bench/Darwin-180B-RSI")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("FINAL-Bench/Darwin-180B-RSI")
model = AutoModelForMultimodalLM.from_pretrained("FINAL-Bench/Darwin-180B-RSI", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FINAL-Bench/Darwin-180B-RSI with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FINAL-Bench/Darwin-180B-RSI"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/Darwin-180B-RSI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/FINAL-Bench/Darwin-180B-RSI

SGLang

How to use FINAL-Bench/Darwin-180B-RSI with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FINAL-Bench/Darwin-180B-RSI" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/Darwin-180B-RSI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FINAL-Bench/Darwin-180B-RSI" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/Darwin-180B-RSI",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use FINAL-Bench/Darwin-180B-RSI with Docker Model Runner:
```
docker model run hf.co/FINAL-Bench/Darwin-180B-RSI
```

🚩 Report: Spam

#10

by haebo1 - opened about 1 month ago

Discussion

haebo1

about 1 month ago

•

This comment has been hidden

haebo1 changed discussion status to closed about 1 month ago

SeaWolf-AI

FINAL_Bench org about 1 month ago

Thanks for the report. Our responses:

Accounts. SeaWolf-AI, openfree and seawolf2357 are VIDRAFT accounts. Sunghokim and ginipick are not VIDRAFT employees; they are community members who collaborate with us through our Discord and Hugging Face community. Sunghokim should have stated that relationship when commenting, and we've asked for the disclosure to be added.
Benchmark reporting. Every score we submitted states its protocol (samples, voting rule, thinking budget) in the .eval_results notes and on the model card. On the GPQA Diamond leaderboard, none of the top 14 entries from other publishers states any protocol. Singling out the one entry that discloses its method would penalize disclosure itself. We support protocol labels for every entry and will follow the team's decision.
Darwin paper (2605.14386). There was no selection among candidates on GPQA Diamond. Each merge produces a single child model; we do not generate multiple children and keep the best one. Because published parent scores could not be reproduced (their evaluation settings were not disclosed), we re-evaluated both parents and the child ourselves under identical settings, greedy first and then maj@4 to maj@8, and reported the child's gain over its own parents measured from scratch. The sentence about "final selection" in Phase 2 was ambiguous, and the next revision will describe this procedure explicitly.
Paper 2608.22876. The paper's contribution is an audit method that localizes causality violations in two forward passes (192/192 injected faults detected). It does not depend on who first found the Mamba2 slow-path defect. We found the Zamba2 defect independently and reported it on July 22, without being aware of the earlier report by another contributor (#47246 / #47250, July 10). We will revise the wording and credit that report.

DedeProGames

30 days ago

Update for the team: a second, independent report exists (#10, by @haebo1 ). Its author's comment is now hidden (I don't know by whom), so I won't characterize it. The publisher's reply there is on record, though, and it matters here:

The publisher states that Sunghokim collaborates with them through their Discord and community, and that this relationship should have been disclosed when commenting in defense of the entry.
The publisher supports protocol labels for every leaderboard entry. That is what this report asks for: label or separate maj@k from single-sample results.
The publisher says the parent models were re-evaluated under identical settings for the Darwin paper. Then the parent's GPQA Diamond score under this model's maj@16 protocol should be easy to publish. It is still missing from the card, and it is the only way to separate the effect of the method from the effect of voting.
The publisher acknowledges the "final selection" wording in the paper was ambiguous and will be revised.

"Other entries don't state a protocol" is an argument for requiring labels on all entries, not for leaving a maj@16 entry ranked #1 without one.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment