MetaDrive-RL policy checkpoints
High-level driving policies for closed-loop evaluation in NVIDIA AlpaSim, from the metadrive-rl project (SFT → DPO → GRPO/RLVR → AIS on 4× Quadro RTX 8000).
The policy does not steer. Every 0.5 s it picks one of 12 high-level actions
(CRUISE@{5,10,15}, FOLLOW, YIELD, BRAKE, CHANGE_LANE_{LEFT,RIGHT}@{5,10,15});
a deterministic planner turns the action into a trajectory and AlpaSim's controller tracks it.
Inputs are a frozen DINOv2-base embedding of the front camera (current + 2 previous decisions)
and deployable state (speed, route command, committed lane offset, route geometry ahead).
Checkpoints
| File | What it is |
|---|---|
checkpoints/sft_dinov2base_ctx3_seed0.pt |
SFT policy head: per-action ADE regression on expert (human-log) decisions, 62 training scenes. Seed fixed in advance. |
checkpoints/grpo_main_it8.pt |
GRPO from the SFT head: 8 iterations, groups of 4 closed-loop rollouts, 62 training scenes, reward = AlpaSim score + 0.2 × progress, KL to SFT. |
checkpoints/ais/{frame,stack,gru,ais}_seed0.pt |
Offline AIS study: memoryless, frame-stack, GRU, and AIS-GRU (transition head) history encoders, matched budget. |
Each .pt is a plain torch.save dict (weights_only=True loadable). The SFT/GRPO files hold
state_dict, head_config, encoder (dinov2-base), context (3) and version.
Use
With AlpaSim and the metadrive-rl plugin installed:
# policy.yaml
policy: {type: neural, checkpoint: /path/to/grpo_main_it8.pt}
greedy: true
decision_period_s: 0.5
scripts/serve_gsplat_renderer.sh 46100 1
scripts/evaluate.sh policy.yaml val_cl runs/eval/my_run 2
python -m metadrive_rl.evaluation.summarize runs/eval/my_run
Results (measured; see results/)
Offline imitation (val regret: ADE of the chosen action minus the best, 4 val scenes, 3 seeds): always-FOLLOW 0.801 m; SFT with DINOv2-base + 3-decision context 0.680 ± 0.023 m; smaller encoder or no images: no gain over the constant action.
Closed loop (AlpaSim, CATK reactive traffic, 16 held-out scenes, 2 rollouts each; scene-level bootstrap CIs; matched scenes): rule 0.449, always-FOLLOW 0.490, SFT 0.504, GRPO-it8 0.549 mean score, all with CIs of roughly ±0.27. Paired GRPO-it8 − SFT: +0.045 [0.000, +0.136], driven by one scene. No policy is distinguishable at this sample size; from paired per-scene differences, resolving ±0.05 needs ~100 scenes.
AIS study: no history encoder separates from the memoryless baseline; AIS (λ=0.3) matches a plain GRU. On ambiguous frames memory models are slightly better (within seed noise).
Data
data/expert_v2/: one row per expert decision (scene, rollout, time, deployable state features,
per-action ADE to the logged future, label) with DINOv2-base features; data/splits.yaml: frozen
scene splits. Images are not included.
Limitations
- Trained and evaluated only in AlpaSim on NuRec reconstructions rendered by a gsplat reimplementation (no camera response model, flat sky); renders degrade when the ego leaves the recorded corridor, where AlpaSim also stops scoring.
- CATK reactive traffic is stochastic; single rollouts are not comparable.
- Simulator-derived preferences and rewards are not human feedback and not real-world safety.
License
Derived from NVIDIA's PhysicalAI-Autonomous-Vehicles-NuRec scenes, subject to the NVIDIA Autonomous Vehicles NuRec Dataset License. Code: Apache-2.0 (GitHub repository).