alexandre5645/avion-0.1

AVION 0.1

A deterministic perception, vision and media kernel behind one C99 header. It stabilises, levels, tracks, measures time to contact, reads a pulse and emits neuromorphic events from ordinary video, on the GPU, and allocates nothing once it runs. For drones, robots, vehicles and smart cameras, and for the AI pipelines and players behind them.

This is not a model: there are no weights to download here. AVION is the front end a model sits behind. It hands a network a foveated NCHW tensor (float32 or float16) with no copy, and it filters what reaches the network: cuts read from the bitstream before decoding, an event stream with the camera's own motion removed, objects with track ids and time to contact.

Site and charts · Whitepaper · Evaluation bundle (8.58 MiB, SHA-256 aa62ffa57a0d362eec9f1ccc83bb88bd48c38310a0c34b06486b18dde2db2898)

Measured head to head

One machine (Intel Core i7-10700, UHD 630), same inputs, and one malloc-counting library preloaded into every process, so an allocation counts the same in AVION, ffmpeg and OpenCV. Lower is better.

Measure	AVION 0.1	Baseline	Baseline tool
Heap allocations per 1,000 frames, 4K HEVC scan	0	132,292	ffmpeg 8.0.1, decode + scdet
Heap allocations per 1,000 frames, motion	0	20,000	OpenCV 4.10.0 DIS ultrafast
Heap allocations per 1,000 frames, survey	0	52,000	OpenCV Scharr + block means
CPU per 4K HEVC frame to find cuts	0.23 µs	31.1 ms	ffmpeg decode + scdet, one thread (134,501×)
Motion estimation, 640×480, one core	0.51 ms	4.35 ms	OpenCV DIS ultrafast (8.5×)
Structure-tensor survey, 640×480	0.179 ms	0.779 ms	OpenCV (4.3×)
Player CPU while playing, 720p	8.44 %	23.90 % · 7.66 %	ffplay · VLC: VLC uses less
Player CPU while paused	0.00 %	0.16 % · 0.00 %	ffplay · VLC (started paused)

The scan rows compare reading slice headers with decoding every pixel: that is the point of pre-decode inspection, not a decoder race. AVION's motion step does more than DIS (time to contact, saliency, objects) on 8×8-sample tiles, not per pixel. Cut accuracy on LG's 4K HDR reel, scored against scdet: 3 of 3 found, 1 extra. Method and raw lines: benchmarks.

What it does, with the figures that check it

Autonomy, drones and robotics

• Virtual IMU: the camera's shake from the background alone, 0.068 px rms off the truth (no gyro).
• Second-order stabilisation: 14.73 → 0.92 px rms on screen (the filter predicted 0.94).
• Auto-horizon: synthetic tilts within 0.06°, real footage within 0.38°; black-bar matte +0.014 ms on the GPU.
• Time to contact with ground-plane and surf rejection: 55 of 55 frames, 3.4 % median error; four objects with crossing-safe track ids; an audio beam 10.3 dB down off-axis that follows the view.

AI front end and neuromorphic perception

• Bitstream perception: scene cuts and activity from H.264 / HEVC slice headers in 0.23 µs a 4K frame.
• Tensor bridge: NCHW float32 / float16, foveated on the target, HDR tone-mapped as shown; windowed and host exports identical (0.000000).
• Event camera (DVS emulation), stabilised: a still scene under shake fires 141 events instead of 1,201,527.
• Remote vibrometry and rPPG: 72.00 BPM through shake (138 if the shake is ignored), 72.00 BPM from Cr chroma alone; an 8.000 Hz vibration read from a ±0.3-sample motion.

Media

• 4K60 HDR on the GPU; 0.00 % CPU paused, 8.44 % playing 720p; a 5.4 KB (1,356-parameter) neural upscaler with exact temporal tile reuse; shadow lift; hum and drone notches −41.8 to −48.5 dB; pinned A–B loop turns in 0.19–3.76 ms; intent control in nine languages; forensic BMP + JSON snapshots.

Real engine output

Each picture is a snapshot AVION wrote through its C99 SDK; the overlays come from the snapshot's sidecar. Footage: Big Buck Bunny © Blender Foundation, CC BY 3.0, or a generated scene whose truth is known.

Plate	Truth	AVION read
Harbour, tilted	+4.00°	+4.06° (+0.004° untilted)
Buoy closing at 1 m/s	TTC 1.33 s	1.24 s; 3.8 % median over 24 frames
Pulse under shake	72.00 BPM	71.98 BPM; Cr only 72.00 BPM
Virtual IMU, same clip	the applied shake	0.031 px rms off it
Event camera, shaken cartoon	—	17.2 M events → 1.71 M stabilised

Quickstart (C99)

A camera in; perception, a tensor and event spikes out. This is the core of quickstart.c, which builds with /W4 /WX and runs against this release:

media_engine_config_t config;
media_engine_config_init(&config);
config.playback = MEDIA_PLAYBACK_HOST;        /* The host drives: live sensors, any platform. */
config.modules = MEDIA_MODULE_TELEMETRY;      /* Perception: flow, TTC, objects, vibrometry. */
media_engine_t* engine = NULL;
media_engine_create(&config, storage, media_engine_storage_bytes(), &engine);

media_sensor_config_t sensor;
media_sensor_config_init(&sensor);
sensor.width = W; sensor.height = H;          /* The arena is committed and sealed here. */
media_engine_open_sensor(engine, &sensor);

media_sensor_frame_t frame = {0};
frame.struct_size = sizeof frame; frame.format = MEDIA_PIXEL_NV12;
frame.width = W; frame.height = H;
frame.planes[0] = luma; frame.planes[1] = chroma;
frame.strides[0] = frame.strides[1] = W;

for (unsigned f = 0; f < FRAMES; ++f) {
    make_frame(f);                            /* Your camera's NV12 frame. */
    frame.timestamp_100ns = (int64_t)f * 333333;
    media_engine_ingest_video_frame(engine, &frame);   /* No allocation. */
    if (f == 0) {                             /* Once a picture is held. */
        media_event_camera_config_t ec;
        media_event_camera_config_init(&ec);  /* 15 % contrast, stabilised. */
        media_engine_set_event_camera(engine, &ec);
    }
    t.struct_size = sizeof t;
    if (media_query_frame_telemetry(engine, &t) == MEDIA_OK && (t.flags & MEDIA_TELEMETRY_LOOMING))
        ++looming;                            /* t.ttc_seconds, t.hazard_x/y, t.objects[] */

    uint32_t n = 0;                           /* Neuromorphic ON/OFF spikes since the last poll. */
    while (media_poll_event_spikes(engine, spikes, 4096, &n) == MEDIA_OK && n) events += n;
}

media_tensor_request_t request;               /* 224 x 224, RGB planes, float32 in [0, 1], the fovea. */
media_tensor_request_init(&request);
request.normalize = MEDIA_TENSOR_IMAGENET;    /* Ready for a torchvision-style model. */
info.struct_size = sizeof info;
media_export_foveated_tensor(engine, &request, tensor, sizeof tensor, &info);

media_engine_close(engine);
media_engine_destroy(engine);

Verification

MSVC x64 /W4 /WX: Release 210/210 passed; Debug 210/210 passed, CTest serial, with a browser decoding video on the same iGPU. GCC with ASan + UBSan: 96/96 passed. 0 third-party libraries: Win32, COM, Direct3D 12, DXGI, WASAPI and C++20.

Limits

• Windows playback is fully exercised; host mode passes on Linux; Android and Apple toolchains have not been run.
• MP4 with H.264 or HEVC, AAC or PCM; AV1 is refused.
• Stabilisation is one translation and one rotation per frame (no rolling-shutter or parallax correction).
• A horizon needs level lines: Big Buck Bunny's meadow, tilted 4°, reads +8.53°.
• Cut accuracy on two files is a sanity check, not a recall figure.

Download

AVION-0.1-Evaluation.zip: the C99 header and SDK guide, the Release demo and static libraries (Windows x64), the head-to-head's raw data and scripts, and the Release suite's log. 8,995,375 bytes, SHA-256 aa62ffa57a0d362eec9f1ccc83bb88bd48c38310a0c34b06486b18dde2db2898 (manifest; digests for integrity, not a signature). A licence for the bundle has not been published yet.