bingaochen/Astra-on-RoboMME-Monitor

Astra-on-RoboMME Monitor

A LoRA adapter for Qwen3-VL-4B-Instruct that monitors whether the current robotic subgoal is complete from causal visual observations. This is the current pipeline’s V6 checkpoint-2246 used by Astra-on-RoboMME. It outputs exactly true or false; it does not generate robot actions.

Files and base model

• adapter_model.safetensors: the original trained adapter tensors (132,195,448 bytes).
• adapter_config.json: LoRA configuration; the machine-specific base-model path was replaced with Qwen/Qwen3-VL-4B-Instruct. Tensor bytes are unchanged.
• additional_config.json: preserved MS-Swift adapter metadata.
• SHA256SUMS: checksums of those three inference files.

The base model is not included. Download it from Qwen/Qwen3-VL-4B-Instruct. The inference package pins base revision ebb281ec70b05090aa6165b016eac8ec08e71b17 at packaging time. The historical GCP base snapshot has not been independently compared byte for byte with that revision.

Exact input contract

Each request contains the task name, natural-language task instruction, current grounded subgoal, and 10 images in this order:

1. Eight front-camera observations, oldest to newest, sampled every three environment steps: max(0, t-21), max(0, t-18), ..., t.
2. The front-camera observation when the current command instance was issued.
3. The current wrist-camera observation.

Early windows repeat execution frame zero where necessary. Reissuing even the same command text starts a new command instance and reference image. There is no explicit command duration, overall timestep, completed-subgoal list or relative frame-offset text in the monitor input. The text templates and image ordering must match examples/champ/input_contract.py in the code repository.

For most tasks, true means the specified current subgoal is complete. For StopCube, a waiting (remain static) subgoal has a different target: true means the cube is on an incoming pass, approximately 16–32 environment steps before reaching the target. Preparation and button-press subgoals still use completion semantics. The monitor does not count passes or decide when to press.

Download and load

Install the simulator/monitor environment using the code setup instructions. The original inference stack uses Python 3.11, PyTorch 2.9.1, Transformers 4.57.3, MS-Swift 3.11.1, PEFT 0.18.1 and FlashAttention 2.8.3. Keep it separate from the JAX/VLA environment. A compatible CUDA GPU and FlashAttention build are required.

from huggingface_hub import snapshot_download

adapter = snapshot_download(
    repo_id="bingaochen/Astra-on-RoboMME-Monitor",
    local_dir="checkpoints/Astra-on-RoboMME-Monitor",
)
base = snapshot_download(
    repo_id="Qwen/Qwen3-VL-4B-Instruct",
    revision="ebb281ec70b05090aa6165b016eac8ec08e71b17",
    local_dir="checkpoints/Qwen3-VL-4B-Instruct",
)

For a frozen evaluation, use the monitor revision pinned in the code repository's examples/champ/weights.json rather than an unpinned latest download. That manifest also verifies all adapter file hashes.

From the cloned code repository root, load the original inference implementation:

import os
import sys
os.environ["USE_HF"] = "1"
os.environ["IMAGE_MAX_TOKEN_NUM"] = "128"
sys.path.insert(0, "examples/champ")
from runner import Monitor

monitor = Monitor(base=base, adapter=adapter)

This applies the original patch-embedding fix and loads the adapter through MS-Swift PtEngine, with bf16, FlashAttention 2, max_tokens=8, and temperature 0. Monitor.predict(...) builds the causal ten-image request, saves the input and response, and strictly parses the boolean output. Use the supplied runner for closed-loop evaluation. A generic chat prompt or default Transformers pipeline is not a validated substitute for this input/decoding contract.

Training

Setting	Value
Base	Qwen3-VL-4B-Instruct; fresh LoRA
Checkpoint	Final step 2246, not validation-selected
LoRA	Rank 16, alpha 32; language-model linear layers
Vision encoder / aligner	Frozen
Learning rate	1e-4
Epochs	2
Effective batch	64 = 8 GPUs × microbatch 4 × accumulation 2
Seed / data seed	42 / 42
Training samples	71,835
Held-out demonstration samples	7,029

Training uses recorded RoboMME demonstration-stage supervision. Ambiguous stage boundaries and conflicting identical visual inputs are excluded. Terminal-stage labels use recorded task-list completion; StopCube waiting labels use approach events. The held-out demonstration split is distinct from the benchmark val split used for closed-loop evaluation. The code release contains training settings and the input contract; raw-H5-to-V6 dataset regeneration is not yet fully packaged.

Pipeline evaluation

Follow the code repository README to reproduce the current pipeline on the official test split using your own GPT API credentials. All task prompts and evaluation commands are included. Final benchmark results are not reported in this model card. Success is determined by the simulator, never by this model's boolean output.

Limits and verification

The adapter was downloaded from the completed training backup and its hashes verified. The portable pipeline passed 48 CPU tests, including causal-input and control-flow checks. A clean Linux GPU installation and end-to-end inference from this hosted copy have not yet been run. This monitor is specialized to the recorded RoboMME camera geometry, commands and task distribution; generalization to other robots or camera layouts is not established.

License and attribution

Apache-2.0, matching this model repository's license declaration. The Qwen base, RoboMME environment and pretrained VLA retain their respective upstream licenses and attribution. See the code repository for those dependencies and citations.