timmyboy26/soundgen-quick-litert-gpu

SoundGen Quick decoder for the LiteRT GPU

The SoundGen Quick audio decoder (autoencoder_model.tflite from jegly/audio at commit e50bf9d, SHA-256 e5483ee50915af1ac419fce5206f14b55572176fcb9cf327ef48432d4c5feba2), rewritten so that it compiles for the GPU with LiteRT 2.3.0. Used by LLM Hub.

LiteRT 2.3.0 does not compile the original decoder for the GPU: its largest dilated convolutions fall back to TFLite's built-in CPU kernel, whose dilated im2col buffer overflows (Dilated im2col buffer size overflowed, CONV_2D node 367). LiteRT 2.1.6 delegated the whole decoder to OpenCL.

Each rewritten CONV_2D with kernel size k and dilation d is replaced by an undilated CONV_2D with kernel size (k − 1)·d + 1, whose taps are d apart with zeros between them. This computes the same output. Nothing else in the graph or weights changes.

File	Rewritten convolutions	Size
autoencoder_model_undilated_1g.tflite	The 2 whose im2col buffer is 1 GiB or more (nodes 367, 390; 128 channels)	317 MB
autoencoder_model_undilated_256m.tflite	The 6 of 256 MiB or more (nodes 207–390; 128–256 channels)	336 MB
autoencoder_model_undilated_all.tflite	All 10 dilated convolutions	651 MB

Licensing follows the upstream model.