esp32-tinylm-flash-unplugged

An ablation of esp32-tinylm: identical weights except the 25M-parameter Per-Layer-Embedding table — the part that physically lives in the ESP32's flash chip — is zeroed. The flash, unplugged.

The intact model tells coherent children's stories; this one collapses into time there time there time... within a few tokens. Side by side, the pair is a direct picture of what a memory chip contributes to a language model's thinking.

Serve both under brainscope's live microscope:

pip install brainscope
python examples/esp32/serve.py                    # intact
python examples/esp32/serve.py --flash-unplugged  # this model

See brainscope's ESP32 example. Original model and hardware: slvDev/esp32-ai (MIT).