LADY — multi-frame RWKV-7 agent for NAVSIM

Checkpoint for LADY, an end-to-end driving agent built on the RWKV-7 / LICA backbone with a diffusion-based trajectory decoder, evaluated on NAVSIM (OpenScene / nuPlan data).

This repository contains weights only. Training code, the agent implementation, and the full evaluation guide live in the LADY NAVSIM devkit.

• Paper: LADY: Efficient End-to-End Driving with RWKV-7 and Linear Cross-Attention
• Code: the navsim/agents/rwkv7_mf/ agent of the LADY NAVSIM devkit

Files

File	Description
lady_state_dict.ckpt	PyTorch Lightning state_dict of the trained agent (epoch 10, global step 58520, 1324 tensors), ~277 MB

The optimizer state was stripped. The file contains {"state_dict", "epoch", "global_step", "pytorch-lightning_version"} and loads exactly like the original lady.ckpt (the agent only reads ["state_dict"]), but it cannot be used to resume training since optimizer momentum is gone.

Usage

1. Install the devkit

Follow the installation guide of the LADY NAVSIM devkit (Python 3.10, torch 2.6.0+cu124, Triton 3.2.0, FLA 0.4.1 pinned to a specific commit; do not upgrade FLA, Transformers, or Triton independently).

2. Download the weights

hf download <USER>/lady-navsim lady_state_dict.ckpt \
  --local-dir "$NAVSIM_DEVKIT_ROOT/base_line"

export NAVSIM_DEVKIT_ROOT=/path/to/RWKV-navsim
export NAVSIM_EXP_ROOT=/path/to/exp
export OPENSCENE_DATA_ROOT=/path/to/openscene-v1.1
export NUPLAN_MAPS_ROOT=/path/to/openscene-v1.1/maps
export NUPLAN_MAP_VERSION=nuplan-maps-v1.0

# architecture selector: 'default' == the architecture this checkpoint was trained with
export MODEL_TYPE=default

3. Score on navtest

export EVAL_FRAMES=10   # runtime history length; see "variable-length evaluation" below

python "$NAVSIM_DEVKIT_ROOT/navsim/planning/script/run_pdm_score_rwkv.py" \
  agent=rwkv7_mf_agent \
  agent.checkpoint_path="$NAVSIM_DEVKIT_ROOT/base_line/lady_state_dict.ckpt" \
  +agent.config.cam_seq_len="$EVAL_FRAMES" \
  +agent.config.lidar_seq_len="$EVAL_FRAMES" \
  train_test_split=navtest \
  train_test_split.scene_filter.num_history_frames="$EVAL_FRAMES" \
  metric_cache_path="$NAVSIM_EXP_ROOT/metric_cache" \
  worker=sequential \
  enable_padding=false \
  experiment_name="lady_navtest_${EVAL_FRAMES}frames"

The NAVSIM metric cache must be built first (see the devkit's evaluation guide). Evaluation writes a per-scenario CSV and an average row to the experiment directory; check the failed-scenario count as well as the average PDMS.

Architecture and training settings

Setting	Value
Backbone	RWKV-7 with LICA linear cross-attention
Decoder	Diffusion-based trajectory decoder, 2 layers, 2 denoising steps
Camera / LiDAR input length	10 / 10
Camera and LiDAR encoders	ResNet-34
Decoder feature dimension	256
Trajectory anchors	100
Future trajectory	8 poses over 4 seconds
Training split	navtrain
Epoch / global step	10 / 58520
MODEL_TYPE	default

Parameters vary with runtime sequence length; the list above is the configuration used for the reported result. Always keep agent.config.use_gt_eval=false for standard evaluation — enabling it makes candidate selection use the ground-truth evaluator, which is an oracle / best-of-N analysis rather than the learned result.

Variable-length evaluation

Training and evaluation history lengths do not have to match. The same checkpoint can be evaluated at 1, 4, 10, 15, 20, or any other positive history length by changing EVAL_FRAMES (which sets cam_seq_len, lidar_seq_len, and the scene-filter history together). No retraining and no cache rebuild is needed because evaluation reads raw sensor data. See the devkit's docs/metrics.md#evaluating-lady-with-n-input-frames for the full protocol and a multi-length sweep.

Results

Metrics as reported in the paper — not measurements from this repository:

Benchmark	Setting	Metric	Result
NAVSIM navtest	Multi-frame LADY	PDMS ↑	90.9
Bench2Drive	LADY with long-term recurrent context	Driving Score ↑	65.12

The temporal-context ablation reports 90.6 PDMS at 10 input frames and 90.9 PDMS at 15 input frames for a model trained with a fixed 10-frame history, which is the setting of this checkpoint.

Reproducing the PDMS number requires the OpenScene / nuPlan data and the NAVSIM metric cache. This checkpoint has been verified to load strictly into the default 100-anchor model (all 1324 state entries matched, all parameters and buffers finite), but dataset-based PDMS reproduction was not performed in the environment where the release was prepared.

License

Devkit code is Apache-2.0. The weights were trained on OpenScene / nuPlan data; review the nuPlan dataset license before use. Trajectory anchors are derived from that data.