Aero-VPA: vision-based quadrotor navigation policies
Aero-VPA is an autonomous navigation framework incorporating a JEPA-based latent world model. The JEPA predictor serves as the latent dynamics world model predicting forward state transitions, while a flow-matching policy acts as the downstream controller. At deployment a deterministic onboard safety shield (15 s depth memory, barrier braking, breadcrumb backoff; no map) filters every setpoint.
These are the checkpoints of the closed-loop evaluation in PX4 SITL + Gazebo Harmonic. Code, training and evaluation scripts: https://github.com/ylee-sling/Aero-VPA. Evaluation records, traces and worlds: ylee-sling/aero-vpa-benchmark. Method: doi:10.5281/zenodo.23633375.
Results in simulation (180 test missions per checkpoint)
Generated from the evaluation records in the dataset by scripts/release/package_release.py (2026-11-03, code c420bf6).
36 test worlds (seeds 2000-2011 × sparse, medium, dense) × 5 missions; every checkpoint, with and without the
shield, flew the same missions, so the comparisons are paired. Crash = collision (privileged distance below
0.35 m) or ground contact.
| checkpoint | success, no shield | success, shield [95% CI] | crashes, no shield → shield | timeouts, shield | paired McNemar p (success) |
|---|---|---|---|---|---|
| r1 | 38/180 (21.1%) | 94/180 (52.2%) [45.0, 59.4] | 72.2% → 22.2% | 23.3% | 1.20 × 10-10 |
| kw | 28/180 (15.6%) | 90/180 (50.0%) [42.8, 57.2] | 80.6% → 26.1% | 21.7% | 4.69 × 10-12 |
| sd | 28/180 (15.6%) | 82/180 (45.6%) [38.4, 52.8] | 76.1% → 35.0% | 16.1% | 4.18 × 10-11 |
| nomotion | 24/180 (13.3%) | 81/180 (45.0%) [37.9, 52.3] | 77.2% → 42.8% | 10.0% | 1.33 × 10-10 |
| r2 | 23/180 (12.8%) | 77/180 (42.8%) [35.8, 50.1] | 80.0% → 38.3% | 16.7% | 4.18 × 10-11 |
| noact | 20/180 (11.1%) | 79/180 (43.9%) [36.8, 51.2] | 82.2% → 36.1% | 14.4% | 2.45 × 10-12 |
Success by world density (shield / no shield, missions per density in brackets):
| checkpoint | dense | medium | sparse |
|---|---|---|---|
| r1 | 28 / 4 (60) | 29 / 17 (60) | 37 / 17 (60) |
| kw | 22 / 1 (60) | 33 / 10 (60) | 35 / 17 (60) |
| sd | 17 / 2 (60) | 26 / 9 (60) | 39 / 17 (60) |
| nomotion | 17 / 1 (60) | 28 / 6 (60) | 36 / 17 (60) |
| r2 | 21 / 0 (60) | 32 / 9 (60) | 24 / 14 (60) |
| noact | 23 / 2 (60) | 25 / 6 (60) | 31 / 12 (60) |
Missions the shield was not revised on (mission 1 and later of every world; the shield pilots flew mission 0):
| checkpoint | missions | success, no shield | success, shield [95% CI] | paired p |
|---|---|---|---|---|
| r1 | 144 | 21.5% | 52.1% [44.0, 60.1] | 2.00 × 10-8 |
| kw | 144 | 15.3% | 54.2% [46.0, 62.1] | 2.28 × 10-12 |
| sd | 144 | 16.7% | 45.1% [37.2, 53.3] | 5.26 × 10-8 |
| nomotion | 144 | 15.3% | 46.5% [38.6, 54.7] | 3.74 × 10-8 |
| r2 | 144 | 13.9% | 44.4% [36.6, 52.6] | 2.40 × 10-9 |
| noact | 144 | 10.4% | 43.1% [35.3, 51.2] | 4.32 × 10-10 |
Videos
FPV re-flights, for illustration, of test missions that both checkpoints succeeded on in the benchmark, recorded from the onboard camera (sim/make_demo_videos.sh). media/videos.json lists every re-flight; the tables above come from the benchmark runs only.
Aero-VPA kw + onboard safety shield: dense world s2004, mission 3; success, 3/3 waypoints in 59 s, mean 0.88 m/s
Aero-VPA kw + onboard safety shield: medium world s2003, mission 1; success, 3/3 waypoints in 38 s, mean 1.84 m/s
Aero-VPA kw + onboard safety shield: sparse world s2004, mission 3; success, 3/3 waypoints in 35 s, mean 1.71 m/s
Aero-VPA r1 + onboard safety shield: dense world s2004, mission 3; success, 3/3 waypoints in 38 s, mean 1.28 m/s
Aero-VPA r1 + onboard safety shield: medium world s2003, mission 1; success, 3/3 waypoints in 45 s, mean 1.52 m/s
Aero-VPA r1 + onboard safety shield: sparse world s2004, mission 3; success, 3/3 waypoints in 35 s, mean 1.83 m/s
Checkpoints
| file | name | kind | values stored | calibrated | SHA-256 | description |
|---|---|---|---|---|---|---|
checkpoints/policy_r1_cal_ep.pt |
r1 (recommended) | policy | 18.13M | yes | a6844216bab9… |
DAgger round 1 (flown by the BC policy v2): demonstrations + round 1, all proprioceptive inputs |
checkpoints/policy_r2_cal_ep.pt |
r2 | policy | 18.13M | yes | 88d362043df1… |
DAgger round 2 (flown by r1): demonstrations + rounds 1-2, all inputs |
checkpoints/policy_r3_noact_cal_ep.pt |
noact | policy | 18.13M | yes | 838d7377ff2e… |
demonstrations + DAgger rounds 1-3; last command withheld from the selector and the solver |
checkpoints/policy_r3_nomotion_cal_ep.pt |
nomotion | policy | 18.13M | yes | cd1fa5c73d63… |
demonstrations + DAgger rounds 1-3; last command and velocity withheld |
checkpoints/policy_r3_sd_cal_ep.pt |
sd | policy | 18.13M | yes | a9a00527fadd… |
demonstrations + DAgger rounds 1-3; all inputs, velocity and last command dropped in 50% of the training frames |
checkpoints/policy_r3_kw_cal_ep.pt |
kw | policy | 18.13M | yes | 56a17afbeb1b… |
demonstrations + DAgger rounds 1-3; all inputs, frames weighted by the error of repeating the last command |
checkpoints/policy_cal_ep.pt |
v1 | policy | 18.13M | yes | bee9546b4723… |
behaviour cloning baseline, non-residual stage 1 |
checkpoints/policy_res_cal_ep.pt |
v2 | policy | 18.13M | yes | b4c8b49ddb82… |
behaviour cloning baseline, residual stage 1 (flew DAgger round 1) |
checkpoints/policy_res_noact_cal_ep.pt |
noact-bc | policy | 18.13M | yes | 1be212516fa3… |
behaviour cloning baseline v2 without the last command |
checkpoints/stage1_res.pt |
stage1_res | stage1 | 13.89M | - | 63d85c9b5dd8… |
frozen stage (i) of every policy above except v1: RGB-D ViT encoder and residual JEPA predictor |
checkpoints/stage1.pt |
stage1 | stage1 | 13.89M | - | ce729adefe65… |
non-residual stage (i) of v1 |
r1 is the recommended policy. Every policy file holds all networks of the paper's Table 1 (18.13M trainable
parameters: ViT encoder, JEPA predictor, primitive selector, CFM solver), the frozen statistics and the calibrated
uncertainty thresholds,
and loads with torch.load(..., weights_only=True); the frozen CLIP text tower is not included (null
instruction embedding only). The stage1* files are the frozen stage (i) networks, needed to retrain stage (ii).
Use
git clone https://github.com/ylee-sling/Aero-VPA && cd Aero-VPA && pip install -r requirements.txt
hf download ylee-sling/aero-vpa --include "checkpoints/*" --local-dir .
python3 sim/make_world.py --seed 2000 --count 12 --density sparse medium dense --out sim/worlds/test
sim/run_benchmark.sh --shield r1_shield checkpoints/policy_r1_cal_ep.pt # PX4 SITL + Gazebo, see the README
from pipeline import VPAInferencePipeline
pipe = VPAInferencePipeline.from_checkpoint("checkpoints/policy_r1_cal_ep.pt", latency_bound=0.06, tracking_lag=0.15)
Limitations and intended use
- Research artefacts evaluated in simulation only (PX4 SITL, Gazebo Harmonic, static procedurally generated forests, no wind, no moving obstacles). They have not flown a real aircraft and are not a safety system.
- Mean flight speed is about 1.2 m/s; about half of the missions succeed with the shield.
- Mission outcomes have a large chance component (simulator timing, CFM noise); one training seed per recipe.
- The shield was revised after its first pilot, which flew mission 0 of every test world; the table above also reports the missions after it.
License
Weights: GNU GPL v3.0 only. Code: GPL-3.0-only. For commercial integration, proprietary licensing, or defense adaptation, contact ylee@sling.ai.kr.
Citation
@misc{lee2026aerovpa,
author = {Lee, Yeonseok},
title = {Aero-VPA: Few-Step Flow-Matching Control over Non-Reconstructive JEPA Latents for Agile Flight on Edge Hardware},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.23633375},
url = {https://github.com/ylee-sling/Aero-VPA},
note = {Preprint}
}