260813-junmo-cho-rbm-cjl-box

Robometer reward model fine-tuned on the CJL parcel-box task (RBY1 + XHand1), trained on real-robot inference rollouts. LoRA on top of robometer/Robometer-4B (itself a Qwen/Qwen3-VL-4B-Instruct backbone, architecture RBM).

Produces per-frame progress and success predictions for a trajectory, used as the value signal for RECAP/AWR.

Results

eval reward-alignment Pearson	0.9902 (step 1000)
final train loss	0.0211
steps	1000 (max_steps reached)

Intermediate best checkpoints kept during the run: 0.9872 @ 500, 0.9900 @ 750, 0.9902 @ 1000. Only the final model is published here — the per-step checkpoint directories (~44 GB) are not included.

Training

Base	robometer/Robometer-4B (training.load_from_checkpoint)
Backbone	Qwen/Qwen3-VL-4B-Instruct
Heads trained	progress, success, preference
PEFT	LoRA r=32, alpha=64, dropout=0.05, bias=none
LoRA targets	q_proj k_proj v_proj o_proj gate_proj up_proj down_proj
Attached to	language_model submodule (see peft_target_module.json)
Optimizer	lr 2e-5, warmup_ratio 0.1, weight_decay 0.01
Frames per trajectory	64 (max_frames, cache built at 64)

Data

CJL parcel-box inference sessions (2026-08-12), LeRobot format, videos pre-scaled to 320x192:

0812_jun_inference_20260812_140019
0812_jun_inference_20260812_151245

Stratified split 163 train / 19 eval trajectories, with one anomalous range excluded (140019:60-87, 28 episodes — consecutive failures). dataset_success_cutoff for both splits: 0.95.

Contents

Inference/serving export. Training cannot be resumed from this repo (no optimizer/scheduler state).

adapter_model.safetensors        264 MB   LoRA adapter
custom_heads.safetensors          20 MB   progress / success / preference heads
model-0000{1,2}-of-00002          9.2 GB  full model snapshot
model.safetensors.index.json
adapter_config.json  config.json  config.yaml
peft_target_module.json  trainer_state.json  training_args.bin
dataset_random_state.json

The full snapshot and the adapter are both present; the snapshot already has the adapter's effect baked in, so loading it directly is enough for inference.

Note on adapter_config.json

This checkpoint was produced before the fix, so its base_model_name_or_path is an empty string. PeftModel.from_pretrained therefore cannot resolve the base on its own — pass it explicitly:

PeftModel.from_pretrained(base_model, "<this repo>")   # base_model loaded from
                                                       # robometer/Robometer-4B

Cause: robometer attaches LoRA to a submodule (language_model), and PeftModel.__init__ fills base_model_name_or_path from the wrapped module's name_or_path, which submodules don't carry. Fixed upstream in our fork by setting name_or_path on the module before get_peft_model; checkpoints produced after that carry the correct id.