mlx-community/neural-drive-optiq-mlx

Neural Drive

Every pixel above was generated by the model, 2 denoise steps per frame.

The mlx-optiq build of the Neural Drive world model, quantised for Apple Silicon. The original weights are 523 MB. OptiQ's mixed 4/8-bit method brings that down to 86 MB, which is what makes it fit. Only the weights changed. The model, its training and its behaviour are the original's.

That model is a racing game with no game engine. Every frame is generated. A diffusion model predicts what you would see next, given the last 7 frames and whatever you are pressing. No track. No physics, no renderer. A 131M-parameter model hallucinates the world in response to your arrow keys, on your Mac's GPU.

It ships as an easter egg in mlx-optiq, our mixed-precision quantiser for Apple Silicon. The command is hidden, so it will not show up in optiq --help:

pip install mlx-optiq
optiq game

That downloads this repo once (115 MB) and drops you into it.

Want to try it before installing anything? The same model runs in your browser on WebGPU at codelion/neural-drive.

What is in the repo

file	what
dit_mixed.npz	dynamics. LatentDiT, dim 768 / depth 12 / patch 3, OptiQ mixed 4/8-bit
vae_dec.npz	VAE decoder, fp32. Required: an fp16 decoder NaNs on every generated frame
seeds7.npz	18 real spawn contexts, [18,7,8,24,48]
config.json	architecture

The model

384x192, 6-dim action space [accelerate, brake, left, right, nitro, drift], rectified flow with per-frame diffusion forcing, distilled to a 2-step sampler. Over 10 fps on an M3 Max.

Training data came from SuperTuxKart run headlessly through pystk. Action labels are read straight out of the engine, so they are exact to the frame. The controller that recorded them was set up to drive badly on purpose: injected swerves, hard turns, sustained drift, brake pulses. Those messy frames are where the model learns what each control does. Record only clean laps and the data contains no example of braking mid-corner, so the model has no idea what braking looks like.

k=2 is fixed and there is no setting to change it. The checkpoint is a distilled 2-step student. Other step counts are off-distribution. They degrade badly rather than trading speed for quality.

Caveats

Drive for long enough and the world drifts. Buildings become other buildings. The track reorganises itself. The fp16 original does this too, so it is not a quantisation artifact. Press R for a fresh spawn.

Links

• Full-precision model and PyTorch/ONNX checkpoints: codelion/neural-drive-model
• In-browser WebGPU demo, no install: codelion/neural-drive
• OptiQ: mlx-optiq.com