Maintained in GitHub: these resources are now included in the RoboDemo code repository. This page remains as a compatibility mirror for existing links.
RoboDemo Refreward
The selected 6,000-step, 7,359,877-parameter scorer used to select robot-video references for RoboDemo-c15-wo. Weight size: 29.4 MB.
Architecture and example | Annotation data | Evaluation environment | Project
What It Does
Given an initial RGB observation and an instruction, Refreward ranks eligible robot episodes using semantic, visual and procedural evidence. Query inputs are initial-only; candidate videos provide their full process. The highest score selects the reference, with episode ID breaking ties.
This repository contains the learned scorer, not the frozen feature encoders or the action policy. Use Qwen3-VL-Embedding-2B and DINOv3 ViT-H+ to regenerate features, and RoboDemo-c15-wo to produce actions from the selected video.
Run The Minimal Example
git clone https://github.com/robodemo-2027/RoboDemo.git
cd RoboDemo
# Install the shared evaluation environment; see the link above.
hf download mxk1998/RoboDemo-Refreward model.safetensors config.json \
--local-dir checkpoints/refreward
hf download mxk1998/RoboDemo-Retrieval data/episodes.jsonl \
--repo-type dataset --local-dir data/refreward
export ROBODEMO_DATA_ROOT=/data/IG-10K-Dataset
export ROBODEMO_RETRIEVAL_QWEN=/models/Qwen3-VL-Embedding-2B
export ROBODEMO_RETRIEVAL_DINO=/models/dinov3-vith16plus-pretrain-lvd1689m
python examples/retrieve_reference.py \
--episodes data/refreward/data/episodes.jsonl \
--checkpoint checkpoints/refreward/model.safetensors \
--data-root "$ROBODEMO_DATA_ROOT" \
--query-episode 3806 --candidate-ids 3836 3815 3820 --device cpu
Download benchmark media from Imitator Game
and frozen encoders from
Qwen and
DINOv3,
under their original access terms. CPU extraction is slow; an allocated GPU
can be selected with CUDA_VISIBLE_DEVICES and --device cuda:0.
The example ranks three genuine candidates and writes their IDs, instructions
and scores to outputs/minimal-retrieval.json. It demonstrates the interface,
not the full benchmark candidate pool.
Load The Scorer
From the cloned code repository:
import sys
sys.path.insert(0, "models/robot")
from safetensors.torch import load_file
from refreward.revision import RefRewardRevision
model = RefRewardRevision(balanced=True, joint_prior=True, dino_dim=1280)
model.load_state_dict(load_file("checkpoints/refreward/model.safetensors"), strict=True)
model = model.float().freeze()
config.json records dimensions and encoder identities.
The scorer has been strictly loaded and checked with synthetic CPU inputs.
The new raw-media minimal example has not yet completed an end-to-end
encoder run in this release. Published C15-wo evaluation can reuse the
recorded selections without rerunning the scorer.
Data And Scope
The selected scorer uses all 15,093 retained annotation conditions. Original partition labels are preserved as provenance, not a new held-out evaluation. Large embedding caches are omitted; rebuilding them and using the small published reference maps are described in the code repository. Training entry points are planned for the next release.
Anonymous authors; manuscript under review. Code and scorer are MIT-licensed. Benchmark data and frozen encoders retain their own licenses.
- Downloads last month
- 25