Strg-Alt-Entf-0x00/Moonshine-ESP32-P4

Moonshine ESP32-P4 Models

Pre-converted Moonshine ASR models for native inference on the Espressif ESP32-P4.

Available Models

This repository currently contains the supported Seq2Seq models:

• moonshine-seq2seq-mshn-base-en
• moonshine-seq2seq-mshn-tiny-de
• moonshine-seq2seq-mshn-tiny-en

Each model directory contains encoder and decoder binaries in .mshn format, and the tokenizers in .bin (ESP32) and .json (Python) format.

The model binaries are available in these variants:

• fp32: Floating-point weights
• fp16: Half-precision floating-point weights
• int16: 16-bit integer quantization
• int8: Per-tensor INT8 quantization
• int8ch: Per-channel INT8 quantization

Repository Layout

moonshine-seq2seq-mshn-base-en/
  moonshine-seq2seq-mshn-base-en-encoder-fp32.mshn
  moonshine-seq2seq-mshn-base-en-decoder-fp32.mshn
  moonshine-seq2seq-mshn-base-en-encoder-fp16.mshn
  moonshine-seq2seq-mshn-base-en-decoder-fp16.mshn
  moonshine-seq2seq-mshn-base-en-encoder-int16.mshn
  moonshine-seq2seq-mshn-base-en-decoder-int16.mshn
  moonshine-seq2seq-mshn-base-en-encoder-int8.mshn
  moonshine-seq2seq-mshn-base-en-decoder-int8.mshn
  moonshine-seq2seq-mshn-base-en-encoder-int8ch.mshn
  moonshine-seq2seq-mshn-base-en-decoder-int8ch.mshn
  moonshine-seq2seq-mshn-base-en-tokenizer.json
  moonshine-seq2seq-mshn-base-en-tokenizer.bin

The German and English Tiny models follow the exact same layout. Debug JSON files are intentionally not included.

Usage on ESP32-P4 (PSRAM Limits)

Copy the required model directory contents to the ESP32 SD card. The firmware loads the encoder, decoder, and tokenizer files from the same model directory.

Important Memory Constraints (32MB PSRAM):

• The int8ch and int8 variants of the Tiny models are the only models that currently fit perfectly into the 32MB PSRAM of the ESP32-P4 alongside the OS (leaving ~3.5MB free).
• The base models and fp32 variants are currently too large.
• Input audio should be chunked (< 10s) to prevent memory fragmentation during convolution preprocessing scratch buffer allocations.

The int8ch variant is generally the preferred embedded variant because it minimizes memory usage while retaining per-output-channel scaling accuracy.

Source Models

The binaries are converted from the corresponding Moonshine PyTorch checkpoints using our automated tools. The mshn suffix identifies the ESP32 binary format.

License

Please review and comply with the license terms of the upstream Moonshine model repositories before using or redistributing these files.