neural-baremetal — the neural CPU datapath booting on a bare-metal PC

A multiboot x86-64 kernel (no OS, no libc) whose AArch64 integer datapath is composed from the exact neural units of Quazim0t0/neural-aarch64-units. It boots on QEMU today (make run) and on a real PC via GRUB (make iso, needs xorriso installed).

How the neural units get to bare metal

PyTorch can't run in a freestanding kernel — but it doesn't need to. Every unit is an MLP verified exact over its complete finite input domain, so the unit is a finite function. tools/export_luts.py:

1. loads each .pt from the downloaded units repo,
2. evaluates the net over its full domain (GPU),
3. re-verifies every output against the golden reference (refuses to export on any mismatch),
4. packs the outputs into binary LUTs (~850KB total).

The kernel embeds the LUTs via .incbin and indexes them directly. Bit identity with the verified nets holds by construction — the LUT is the net's complete extensional behavior. This mirrors the project's own methodology: the Space's C fast core is audited against the neural core; here the LUTs are the neural core, exported exactly.

Two ways the neural units run on metal

1. Exact LUTs (default datapath): the tabulated behavior of each verified net, bit-identical by construction. This is what the Linux boot uses.
2. Live MLP inference in assembly (kernel/sdot.S + kernel/mlp.c): the raw fp32 weights (tools/export_weights.py, 4.1MB) evaluated Linear→ReLU→Linear→ReLU→Linear per slice op, dot products in SSE assembly with a fixed evaluation order. tools/host_verify.c compiles the SAME mlp.c+sdot.S and sweeps every unit's complete domain against the LUTs (403,296/403,296 EXACT) — and because fp32 with a fixed instruction sequence is deterministic, that pass transfers to bare metal verbatim. g_alu_mlp=1 routes the whole A64 ALU composition through live inference; the boot self-test demonstrates 64-bit ADDS/MUL/bitmask computed this way.

Layout

tools/export_luts.py   .pt -> exact LUTs (+ luts/luts.h)
luts/                  generated binary tables
boot/boot.S            multiboot1 header, 32->64-bit long-mode trampoline
boot/luts.S            .incbin the tables into .rodata
boot/linker.ld
kernel/neural_alu.c    A64ALU composition, ported from arm64_units.py:
                       ADC8/SBB8 ripple, logic/NOT slices, 1-bit shift
                       slices, MASK8+ADC multiply trees, restoring
                       division, CONDEXEC, DECODE_BITMASK
kernel/kmain.c         16550 UART console + self-test vs an independent
                       golden oracle (same pattern as verify_a64_alu)

Run

make            # exports LUTs if needed, builds neural.elf
make run        # boots in QEMU, self-test output on serial, exits clean
make iso        # GRUB-bootable ISO for real hardware (needs xorriso)

Current output: CONDEXEC full-domain sweep, randomized 32/64-bit ALU composition tests (adds/subs + NZCV, logic, shifts, mul lo/hi, udiv), DECODE_BITMASK spot checks — ALL TESTS PASS.

Boot menu (neural.iso)

busybox shell        Image + net.dtb + sh.cpio, interactive console
llama.cpp demo       Image + llm.dtb + llm.cpio (TinyStories-260K GGUF)
FULL N/N self-verify complete 403,296-point MLP-vs-LUT sweep on the
                     machine itself, then shell (slow under TCG; ~a
                     minute-ish on real silicon)
AUDIT digest         bounded 5M-instruction boot emitting an FNV-1a
                     digest of per-instruction (pc, regfile-hash)
                     records; tools/audit_host.sh prints the reference
                     digest from a native host run of the same core.
                     Confirmed matching: 0x166198a43829b712.

The ISO is BIOS/El Torito (grub-mkimage + genisoimage; no xorriso needed) and boots in VirtualBox: Other/Unknown (64-bit), >= 1GB RAM, attach neural.iso. Console output mirrors to VGA text and COM1; input from PS/2 keyboard or serial. Cmdline word crashtest exercises the exception handler.

Robustness: IDT with full register dumps on any CPU exception; E820-derived heap bounds (panics informatively if <400MB contiguous available).

Roadmap to "boot it all"

1. [done] datapath on metal — LUT export + composed ALU, self-tested.
2. port the emulator core — neon64.c (the Space's C fast core, ~41M instr/s) is nearly freestanding already; replace its native ALU ops with neural_alu.h calls (or keep native ops and add the audit mode that cross-checks against the LUT datapath, like the Space's Audit tab).
3. guest assets — embed the Space's kernel Image + initrd (GRUB modules keep the kernel binary small), PL011 UART bridged to the 16550.
4. boot Linux — the same busybox/llama.cpp payloads the Space runs, now on a real PC with no host OS underneath.

Done: steps 1-4 (LUT datapath, neon64 port, embedded assets, Linux boots to an interactive shell; llama payload embedded; asm MLP inference; audit).

Done additionally: networking. kernel/e1000.c is a polled Intel 8254x driver (PCI scan, 32-desc rings, no interrupts); port/src/net.inc carries the Space's virtio-net device model verbatim with the user-mode NAT swapped for a raw-frame bridge to the e1000. Under VirtualBox/QEMU NAT the VM's own engine provides the identical slirp topology (guest 10.0.2.15, gw 10.0.2.2, DNS 10.0.2.3) the original NAT emulated. The guest's virtio-net config-space MAC is the physical NIC's MAC. Verified end-to-end with the nettest boot entry: DNS resolved example.com and TCP connected from the guest. (Known quirk: nettest's HTTP GET reported 0 body bytes; DNS/TCP confirmed.)

Done additionally: the DDR5 memory-logic layer (from neural-ddr). tools/export_ddr5_luts.py re-verifies all seven units against the repo's golden functions and tabulates them (ODECC enc/dec, DBI enc/dec, WR_CRC slice, ADDR_MAP, CMD_DECODE). kernel/ddr5.c presents DDR5 on-die-ECC behavior over the machine's actual RAM, per the neural-ddr rule (logic, not invented capacity): data bytes stay in guest RAM, a nibble-packed parallel array holds Hamming(12,8) parity; every guest write runs the neural ECC encoder, every read reassembles the codeword and runs the neural decoder, correcting single-bit faults in place. Boot does DDR5 "memory training" (parity for the loaded image) exactly like real DDR5 init. Hooks live in port/src/neon64.c's raw_ld/raw_st (marked "bare-metal port addition"). Boot words: ddr5 arms the path, ddr5fault additionally flips a random stored bit every 4M reads — Linux keeps running because every fault is corrected on read; stats print at guest power-off. The boot self-test proves all 3328 single-bit fault patterns correct on the machine itself.

All previously-open items are now done:

• framebuffer: multiboot video header requests 800x600x32; GRUB hands over a VBE (BIOS) or GOP (UEFI) linear framebuffer; kernel/fbblit.c blits the guest's simple-framebuffer (0x4F000000, a8r8g8b8) live from the device poll. fbcon verified on-screen via QEMU screendump.
• Ubuntu: ubuntu-con.ext4 embedded; NEON_DISK reads it from the blob-fs into a writable in-RAM copy; ext4 root mounts on virtio-blk (verified).
• SMP: blobs/net-smp2.dtb (generated with pylibfdt: cpu@1 + psci) + NEON_CPUS=2 runs the deterministic round-robin scheduler — "Total of 2 processors activated" verified to shell.
• free(): first-fit free list with coalescing replaces the pure bump allocator.
• timekeeping: TSC calibrated against PIT ch2 at boot; nanosleep is a real TSC deadline wait.
• keyboard: E0 extended scancodes (arrows/home/end/del/pgup/pgdn/insert) inject ANSI escape sequences; right-ctrl handled.
• UEFI: hybrid El Torito ISO (BIOS eltorito.img + FAT ESP with grubx64.efi, built with grub-mkimage + mtools + genisoimage). Verified booting under OVMF, including the fragmented UEFI E820 map and a GOP framebuffer.

Boot menu (13 entries): shell, llama, fullverify, audit, nettest, ddr5, ddr5+fault, fbcon, fbcon+llama, ubuntu, smp2, ubuntu+nstore+faults.

neural-storage + DaisyChain-Web integrations

• Self-healing disk (neural-storage): kernel/nstore.c stripes the guest's virtio-blk disk RS(4+2) over GF(2^8); the GF multiply table is the composed neural LOG/EXP units, re-verified 65536/65536 at export (tools/export_storage_gpu_luts.py). Per-sector FNV checksums locate corruption; an incremental scrub (polled from the emulator's device loop) reconstructs any <=2 lost shards per stripe via Cauchy-matrix inversion — a faithful C port of storage/rs.py. Boot words: nstore arms it, diskfault injects random disk corruption that the scrub heals live.
• vGPU (DaisyChain-Web GPU emulation): kernel/vgpu.c is the verified INT8 compute pipeline — mul8/requant/relu LUTs (re-verified at export), lutMatmul (exact), an SSE2 pmaddwd backend that is PROVEN bit-identical to the LUT at init before being trusted (webgpu.js's DP4A discipline), the 3xINT8 hi/lo GEMM (near-fp32 from 8-bit units), and the B2B MLP chain (mul->requant->relu->mul). The boot self-test checks the chain against golden integer math and GF(256) field axioms.