longnav-objectnav-flow-pose-2p5hz

Continuous-control ObjectNav policy at a 2.5 Hz observation rate, trained for 9,600 steps (485k observation-turns of supervision). Flow-matching action head, pose injection, LoRA over Qwen3-VL-2B.

What this is

A continuous-control ObjectNav policy: a Qwen3-VL-2B backbone with LoRA adapters and a flow-matching action head that emits a timed trajectory -- a chunk of 20 cumulative relative planar poses [dx, dy, dtheta] spaced 0.04 s apart -- which a PID controller tracks on a holonomic mobile base inside a Bullet-simulated Habitat scene.

This differs from mainstream ObjectNav agents in two ways worth stating plainly:

• Most ObjectNav policies emit discrete actions (MOVE_FORWARD 0.25 m, TURN 30 deg) executed by kinematic teleport. This one emits a timed trajectory executed against real base dynamics (rigid body, contact, acceleration limits).
• It reads a pose modality. Each observation carries the agent's planar pose, converted to the frame of the first observation in context by relative_se2 and injected at a <pose> marker token.

Trained on human ObjectNav demonstrations (Habitat-web) replayed as continuous robot trajectories.

Files

file	what
adapter/	LoRA adapters (r=128, alpha=256) over Qwen/Qwen3-VL-2B-Instruct
turn_vector_head.pt	flow-matching action head + pose encoder
turn_vector_head_config.json	head config, including the <pose> modality spec
trainer_state.json	full training curve, for provenance
tokenizer / preprocessor files	as trained

The optimizer state is not included -- these are inference checkpoints, not resume points.

The one setting that will silently ruin your numbers

--dt is 0.04 for every model in this family; the tick rate never changes. What changes is --gap, the number of ticks executed per observation, and it must match the corpus the model was trained on:

model	observation rate	--gap	chunk length
longnav-objectnav-flow-pose-2p5hz	2.5 Hz	10	20
longnav-objectnav-flow-pose-1hz	1 Hz	25	30

Using --gap 5 here executes a fraction of each chunk and discards the rest. It does not error. It presents as a slow, weak policy.

Evaluating

Evaluation needs the harness in habitat_physical_nav, because the simulator and the model cannot share a Python environment -- habitat_sim and transformers+flash-attn have conflicting dependency sets. The harness runs the policy in a second interpreter over a unix socket, which is what --policy-python is for.

python scripts/eval_objectnav_policy.py \
  --episodes  <hm3d objectnav val split> \
  --scene-root <scene_datasets> \
  --output-dir <out> \
  --ckpt <this checkpoint> \
  --policy-python <env with transformers> \
  --policy-sys-path <longnav src> \
  --policy-backend flow_rollout --policy-seed 1000 \
  --max-steps 175 --dt 0.04 --gap 10 \
  --success-distance 1.0 --distance-to VIEW_POINTS \
  --seed 0 --navmesh dataset --fresh-sim-per-episode \
  --auto-stop --auto-stop-delay 15 \
  --record-video

--max-steps 175 with --gap 10 is 70 s of simulated time, which is how the numbers below were produced. Budget is in policy steps, so the same --max-steps buys different amounts of time at different --gap; compare models at matched sim time, not matched steps.

--navmesh dataset matters. The alternative (robot) recomputes the navmesh for the robot's footprint, and because the robot is a physics body not constrained to that mesh, snap_point can resolve its proxy across an island boundary and the metric silently starts measuring a different goal instance. Use dataset.

--fresh-sim-per-episode is not optional for comparable numbers: without it, simulator state leaks between episodes.

Results

HM3D ObjectNav val, 101-episode sample, --navmesh dataset, 70 s sim time, success distance 1.0 m.

metric	value
oracle success	0.535
oracle SPL (as reported)	0.334
oracle SPL (corrected, see below)	0.265
NPWPL	0.358
closest approach	2.94 m
path length	23.4 m

Read the oracle metrics, not success/SPL. This checkpoint has no stop classifier. Evaluation runs with --auto-stop, so episode termination is driven by the harness, not by the policy -- success measures whether the agent was still within 1 m when forced termination fired, which is set by --auto-stop-delay. oracle_success (did it ever reach the goal) and NPWPL (progress toward the goal, normalised by start distance, weighted by path length) are the metrics that mean something here.

oracle_spl as reported is overstated. The OracleSPL measure freezes path length incorrectly and returns 1.000 on episodes where the agent travelled further than the geodesic before first reaching the goal. The corrected column recomputes it as start / max(start, path_at_closest_approach), gated on oracle success. Prefer the corrected value.

Limitations

• No stop head, so the policy cannot terminate its own episodes.
• Under-drives relative to the demonstrations it imitates: drive fraction 0.398 against the training corpus's 0.517.
• Trained on MP3D human demonstrations, evaluated on HM3D -- cross-dataset.
• Requires the pose modality. Feeding it images alone will not work; the <pose> marker must be present and the value must be the raw planar pose, with the training-frame conversion done by pose_frame.relative_se2 in the model environment.

Related

• Aasdfip/longnav-objectnav-flow-pose-2p5hz -- 2.5 Hz sibling
• Aasdfip/longnav-objectnav-flow-pose-1hz -- 1 Hz sibling

Decision frequency is the variable these two isolate. A 5 Hz model trained identically reaches oracle success 0.366 / NPWPL 0.266 while consuming more supervision than either of these, and degrades with further training where these improve.