Low-Altitude-3D-LLM: final checkpoints and complete evaluation results

This repository contains the actual final checkpoint files for three HUGE-Bench 5-epoch experiments, plus complete evaluation results for the pi0.5-base reproduction and our final VLA using the fine-tuned Qwen3-VL-2B backbone. Each evaluated model has all 993 videos and trajectories. Results are organized under the respective experiments/pi05_hugebench/ and experiments/low_altitude_3d_llm/ directories.

Training, evaluation, model conversion, and environment setup code: GitHub, feat/starvla-qwen3vl2b-hugebench.

Selected three-way comparison videos

Watch six synchronized GT / pi05 / our-model videos (3 seen and 3 unseen), with official episode identifiers, task instructions, metrics, and links to both original videos in the comparison table.

Watch six side-by-side AirSim comparisons: PI0.5 BASELINE on the left, OUR MODEL on the right, with original English task captions. The 12 individual source replays are also available.

Watch two new AirSim comparisons with synchronized GT, or download the self-contained offline HTML. Each case includes both the original evaluation view and the AirSim replay.

Final checkpoints

Version	Final checkpoint	Contents
Original pi0.5 reproduction, without current-state input	67189	Complete Orbax checkpoint: params, train_state, assets, and metadata; 68 files, 44,747,293,519 bytes
StarVLA-PI + official Qwen3-VL-2B	20260916 final	final_model/pytorch_model.pt and 4 companion configuration files; 16,798 updates
StarVLA-PI + our fine-tuned Qwen3-VL-2B	20260920 final	final_model/pytorch_model.pt and 4 companion configuration files; 16,798 updates

The separate pi0.5 experiment with current-state input is paused and has no final checkpoint in this repository. Base weights and intermediate training checkpoints are outside this upload's scope.

The two StarVLA .pt files are each 5,839,846,728 bytes. Their distinct content checksums are recorded in the artifact inventory.

pi0.5-base reproduction: complete 5-epoch evaluation

Browse all 993 videos and trajectories · Run details

The run pi05_overall_5ep_run1, final checkpoint 67189, was initialized from gs://openpi-assets/checkpoints/pi05_base/params and trained on HUGE-Bench. It corresponds to the pi0.5 final checkpoint already published above. The full evaluation has 576 seen + 417 unseen episodes, with one MP4 video, one NPZ trajectory, one PNG trajectory plot, and one instruction text per episode. The run uses seed 0, four inference shards, and exec_steps=10.

The complete run directory contains 3,985 original files / 3,115,958,494 bytes, including both metric JSON files, run_manifest.yaml, and 10 evaluation logs. The original training and evaluation driver logs are also included under experiments/pi05_hugebench/logs/, bringing this addition to 3,987 source files / 3,119,124,872 bytes. The source is the ll_0909/pi05_hugebench/ Blob backup; the later ll_0923/experiments/pi05_hugebench/ backup contains corresponding results. These are preserved historical outputs; training and evaluation were not rerun for this upload.

Split	Episodes	TCR@1m ↑	TCR@2m ↑	TCR@5m ↑	Avg. TCR ↑	nDTW ↑	NSP ↑	SR ↑	CR ↓	CSPL ↑
Seen	576	38.16	61.13	83.45	60.91	50.47	59.51	83.68	7.29	83.23
Unseen	417	31.38	50.03	71.23	50.88	39.67	55.37	69.06	6.24	68.50
Overall	993	35.31	56.47	78.32	56.70	45.94	57.77	77.54	6.85	77.05

Values are multiplied by 100 and rounded to two decimals, from metrics/metric.json. metrics/paper_aligned.json preserves the separate paper-aligned report and all 993 episode records. The source metrics retain their original absolute filesystem paths.

Our final VLA: complete results and all 993 evaluation videos

Browse evaluation files and videos

• Seen: 576 MP4 files; Unseen: 417 MP4 files.
• Each video shows ground truth on the left and the model prediction on the right.
• Individual MP4 files can be downloaded directly, without extracting an archive.
• Each episode has a VIDEO_COMPLETE.json record. Evaluation configuration, metrics.json, and metrics_table.md are included.
• Video completion was checked independently against all 993 episode records. The older timing note at the end of metrics_table.md describes its original upload; this repository includes the subsequently completed full video set.

Split	TCR@1m ↑	TCR@2m ↑	TCR@5m ↑	Avg. TCR ↑	nDTW ↑	NSP ↑	SR ↑	CR ↓	CSPL ↑
Seen	46.12	67.37	85.27	66.25	54.78	69.34	83.33	2.43	82.03
Unseen	37.58	54.13	72.22	54.64	38.64	67.59	68.35	2.88	67.53

All metrics are multiplied by 100 and rounded to two decimals. The configuration and per-episode data in metrics.json are authoritative.

Complete results directory snapshot (2026-09-28)

The entire experiments/low_altitude_3d_llm/results/ directory is mirrored here, with 6,985 files / 4,784,899,596 bytes. File contents are preserved. Hub run directories use hugebench_low_altitude_3d_llm_5ep_b512_20260920; the manifest records original local paths in source_path.

• 993 MP4 videos and 993 NPZ trajectory files.
• 1,994 PNG images, including trajectory plots and video export images.
• 1,994 JSON files with metrics, configurations, per-episode records, and video completion records.
• 993 TXT files, 15 logs, one metrics Markdown table, and two completion marker files.
• All local files are included, including the evaluation logs and video export records.

The remote path list, sizes, and content hashes were checked against the complete local directory. Sample trajectory, image, and log files were also downloaded without authentication and checked by SHA256.

Download

Install the Hugging Face CLI if needed:

python -m pip install -U huggingface_hub

Run the desired command from your target workspace. The CLI preserves the paths:

# pi0.5 final checkpoint
hf download lld-koi/Low-Altitude-3D-LLM --local-dir . \
  --include 'experiments/pi05_hugebench/artifacts/checkpoints/**'

# StarVLA-PI + official 2B final checkpoint
hf download lld-koi/Low-Altitude-3D-LLM --local-dir . \
  --include 'experiments/starvla_pi_qwen3vl_2b/artifacts/checkpoints/**'

# Our VLA final checkpoint
hf download lld-koi/Low-Altitude-3D-LLM --local-dir . \
  --include 'experiments/low_altitude_3d_llm/artifacts/checkpoints/**'

# Complete results: trajectories, images, videos, metrics, records, and logs
hf download lld-koi/Low-Altitude-3D-LLM --local-dir . \
  --include 'experiments/low_altitude_3d_llm/results/**'

# pi0.5 complete 5-epoch results and training/evaluation logs
hf download lld-koi/Low-Altitude-3D-LLM --local-dir . \
  --include 'experiments/pi05_hugebench/results/trained/pi05_overall_5ep_run1/**' \
            'experiments/pi05_hugebench/logs/**'

# Everything in this repository (about 64.33 GB)
hf download lld-koi/Low-Altitude-3D-LLM --local-dir .

Integrity and provenance

The source-file inventory, checksums, and replay verification records are organized in metadata. The inventory covers all 11,050 original source files; the AirSim replays have a separate checksum list.

The checkpoint formats require the corresponding experiment code linked above. These are complete VLA/Orbax training artifacts, not standalone Transformers AutoModel exports. See the source projects and upstream model cards for their respective licenses and usage terms.