SIMFORGE-D1.5

SIMFORGE-D1.5 is our best distilled checkpoint derived from NVIDIA's Alpamayo 1.5 10B.

Results

SIMFORGE-D1.5 checkpoint

Outcome	Result
Model size	35% smaller than Alpamayo 1.5
Model GPU memory	14.42 GiB peak reservation
Reasoning performance	Parity retained in our internal reasoning evaluation

Matched 32-scenario AlpaSim benchmark

Everything in this table was evaluated on Alpamayo 1.5: the original 10B parent, SIMFORGE-D1.5, derived from it, and the compiled FlashDrive runtime applied to Alpamayo 1.5. All systems ran once on the same 32 scenarios with AlpaSim 0.89.0, the same four-camera observation interface, the same challenge configuration, and full 32/32 coverage.

Metric	Alpamayo 1.5 10B	SIMFORGE-D1.5	Alpamayo 1.5 + FlashDrive
Mean AlpaSim `scene_score` ↑	0.7061	0.6608	0.5928
Pass rate ↑	75.00%	75.00%	62.50%
At-fault collision rate ↓	9.38%	12.50%	9.38%
Off-road rate ↓	15.62%	12.50%	28.12%
Mean route progress ↑	0.7816	0.7308	0.7826
Mean distance to GT trajectory ↓	3.1804 m	2.5007 m	3.0313 m
Coverage	32/32	32/32	32/32
Infrastructure failures	0	0	0
Eight-GPU wall time	1h 29m	47m 21s	43m 51s
Observed model/runtime memory	35.0 GiB max per GPU including AlpaSim services	14.42 GiB model peak	20.81 GiB model peak

SIMFORGE-D1.5 retained 93.6% of Alpamayo 1.5's mean scene_score, matched its 75% pass rate, and had the lowest off-road rate and distance to the ground-truth trajectory.

FlashDrive used compiled max-autotune-no-cudagraphs kernels. Across 6,359 Drive calls it produced zero constant-velocity fallbacks, zero empty trajectories, and zero inference failures. A warmed, neutral-context FlashDrive model trajectory covered 33 initial Drive calls while live-context plans became available; those calls are reported separately as model-bootstrap drives rather than hidden as normal live-context plans.

Raw aggregate results are available in benchmark/benchmark_results.json and benchmark/benchmark_results.csv. Per-scene results and official KPI summaries are under benchmark/per-scene/ and benchmark/summaries/.

Evaluated samples

The benchmark uses a frozen 32-scene subset of the NuRec96 scene collection. Every model in the results table was evaluated once on every scene below. The exact machine-readable manifest is benchmark/scenarios32.csv, with category definitions and membership in benchmark/scenario_categories.json. Manifest SHA256: b3fb55c42be540ed464b28ff175476dd4b0c67375d5e1ff75be8f4f42971c6e3.

Scenario type	Count
`guardrail_offroad_complex_interaction`	8
`lane_offset_or_curb_sensitive`	6
`right_turn_or_turn_after_intersection`	8
`stop_yield_lead_or_pedestrian`	3
`straight_lane_keep`	7

Exact 32 scene IDs

Scene ID	Scenario type
`007a5809-8a56-40b5-8af5-7e0f65229496`	`guardrail_offroad_complex_interaction`
`00eb506e-de3a-407c-8a76-c763de8dc0d8`	`lane_offset_or_curb_sensitive`
`01d503d4-449b-46fc-8d78-9085e70d3554`	`straight_lane_keep`
`0593b1f2-244a-4615-bc55-69be0c80136f`	`guardrail_offroad_complex_interaction`
`06b3e399-a820-49ef-9e69-134cfdb7652a`	`lane_offset_or_curb_sensitive`
`098e2482-6db2-473b-b08a-2a80be9320f3`	`straight_lane_keep`
`0a228e44-2e22-40fa-a4da-9dcded392963`	`right_turn_or_turn_after_intersection`
`0ec1da73-99b4-4f0a-8266-6f696947b162`	`right_turn_or_turn_after_intersection`
`0f525628-b4c3-4506-8b16-a27385913fa5`	`right_turn_or_turn_after_intersection`
`0fd2c051-f5e1-4416-9bb5-9b93d92f55fb`	`straight_lane_keep`
`16150118-eef4-42d0-8358-15e787e236a5`	`stop_yield_lead_or_pedestrian`
`17bec9f8-aa14-4091-ae83-b05714fe6e81`	`right_turn_or_turn_after_intersection`
`19585c5c-d523-49f4-bf15-59291e6c9278`	`guardrail_offroad_complex_interaction`
`1ad2258e-7166-4af6-a076-d2174b78f73a`	`guardrail_offroad_complex_interaction`
`1c5b5611-79ba-43a5-8f73-b597e3620ef9`	`stop_yield_lead_or_pedestrian`
`225eb8de-bf61-4fa9-b4b1-1f749cf8b57f`	`lane_offset_or_curb_sensitive`
`2374aa97-69ec-4365-8735-b051ff0e1886`	`straight_lane_keep`
`2387cbf7-1d05-4854-8c20-d3ed91d1bffe`	`lane_offset_or_curb_sensitive`
`240c081f-03ca-4eae-89a3-1b821e47d502`	`guardrail_offroad_complex_interaction`
`25151bb2-6aca-4b8c-bc42-841a27d32d77`	`right_turn_or_turn_after_intersection`
`2554a11a-f982-438b-804f-96d04aa94903`	`straight_lane_keep`
`26f70650-1e7f-4b35-9e0a-9afe64537bc7`	`guardrail_offroad_complex_interaction`
`28621273-20ed-4570-98d1-cd0dcf451fc1`	`guardrail_offroad_complex_interaction`
`2a412836-cbe9-4e4c-a0e1-302a2959098d`	`guardrail_offroad_complex_interaction`
`2b4b2e84-cff4-4697-a4b5-ccb706f69438`	`lane_offset_or_curb_sensitive`
`2ce64e22-57fe-4d5f-a0ea-2d1825b01ea9`	`straight_lane_keep`
`2e132b04-dca4-4450-ad8f-7e54404fd9d9`	`right_turn_or_turn_after_intersection`
`2e9fe627-4164-4e0d-965a-d10c8d131e81`	`lane_offset_or_curb_sensitive`
`35cec769-9f45-4b14-a70d-b8778732ce0c`	`stop_yield_lead_or_pedestrian`
`36444635-aed6-4f44-bd1e-105cfb15a4b0`	`straight_lane_keep`
`37d660b1-8abc-4a24-8778-94ca50a268c2`	`right_turn_or_turn_after_intersection`
`3a42e0a8-52af-4c18-a676-40c34186e686`	`right_turn_or_turn_after_intersection`

Method

We started with Alpamayo 1.5 and identified parts of the model that could be removed with the least effect on its behavior. We then trained the smaller model to reproduce the original model's driving and reasoning capabilities. The process was repeated in stages, with each version evaluated before selecting this checkpoint as the best balance of size and performance.

Intended use

This is a research checkpoint. It is not a production driving system and should not be used to control a real vehicle or make safety-critical decisions.

License and attribution

SIMFORGE-D1.5 is derived from NVIDIA Alpamayo 1.5. Its model weights are provided under the included OpenMDW 1.1 license. Users are responsible for reviewing the upstream model terms and applicable dataset licenses before use. NVIDIA has not endorsed this release.