Hi @geneing — thanks for trying the model, and for the Pixel angle. That is where these graphs have the most room, so I am glad someone is looking at it.

The build script is now in the zoo, next to the sample app and the recipe notes:

• Script: https://github.com/john-rocky/LiteRT-Models/blob/main/pockettts/scripts/build_pockettts.py — the PyTorch → LiteRT re-authoring of each graph (flow-LM step, fused step + head, Mimi decoder transformer, SEANet decoder), the fp16 pass, and a tflite-vs-eager parity check per stage.
• Recipe notes: https://github.com/john-rocky/LiteRT-Models/tree/main/pockettts — the op-by-op rewrite table, the block-decode layout, and the Pixel 8a measurements.

To reproduce:

pip install torch==2.12.1 litert-torch==0.9.3 ai-edge-litert==2.1.6 ai-edge-quantizer==0.8.0 sentencepiece safetensors huggingface_hub scipy
git clone https://github.com/kyutai-labs/pocket-tts        # tested at 001cf6e
cd LiteRT-Models/pockettts
PYTHONPATH=/path/to/pocket-tts python scripts/build_pockettts.py all   # or one stage: flowlm | head | fused | dectx | deconly | assets | pipeline

It downloads the ungated kyutai/pocket-tts-without-voice-cloning weights and writes every file in this repo into scripts/out/. I re-ran it today: the outputs are byte-identical to the published files (sha256 of all 20 LFS files), and the parity numbers on the card are printed per stage. About 3 minutes on a Mac.

Two things measured on a Pixel 8a (Mali-G715, LiteRT 2.1.6) that may save you time:

• The per-frame cost there is the packed-KV upload plus the dispatch count, not arithmetic: one frame was ~43 ms with the split graphs (11 ms KV write, 12 ms run(), 20 ms readback), and fusing the head into the step graph with one output tensor took 1.1 s off an 8 s utterance — that is why the app runs pt_flowlm_fused. The 25 MB KV round-trip per step is the largest remaining term.
• pt_mimi_dec_tx compiles fully on that GPU, but its output is audibly degraded (alba voicing HNR 0.9 dB vs 2.8 dB on CPU, which matches the fp32 reference), and GpuOptions(precision = FP32) does not recover it — so the app runs that one graph on CPU. A force_gpu.txt containing dectx in the app's files dir puts it back on the GPU for experiments.

If something moves the Pixel numbers, I would be glad to hear about it — and glad to take a PR into the zoo.

Thank you very much!!! LiteRT and onnx conversion from pytorch can be very difficult to do correctly. There are too many places where small errors creep up.

@mlboydaisuke I did a bit of exploration with litert pockettts model on my pixel 10 phone (Tensor G5 processor). Very interesting results.

TLDR: for LM per step GPU synchronization cost is more than compute cost - it makes running on CPU faster. PowerVR GPU is slower than Mali. I was getting only about 0.8x RTF with LM on GPU!

I tried amortizing gpu sync cost by running multiple steps of LLM per invocation (4 and 8 steps). It improved the timing, but CPU was still faster.

I also tested running the model on G5 NPU. mimi_deconly will not compile for NPU - plugin crashed! LM ran almost as fast as on CPU. Big improvement with dec_tx running on NPU - fast and preserved quality.

Optimal configuration LM:CPU DECTX:NPU DEC:GPU. Here are some generated timing tables.

End-to-end per utterance (LM steps 28 prompt + 70 gen)

Micro-benchmark, 128 LM frames. "copy-in" = host→input-tensor write; "readback+sync" = output read, which is where the blocking wait lands.

Decoder stages