Title: MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Control
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: text-to-speech (TTS), speech synthesis, voice cloning, Mamba, state space models (SSM), diffusion TTS, prosody modeling, streaming/low-latency

Abstract: \begin{abstract}
\noindent We present MambaVoiceCloning (MVC), a text-to-speech framework whose encoder/conditioning path is SSM-only at inference: no attention modules and no recurrent units remain at deployment. MVC comprises three linear-time components—a bidirectional Mamba text encoder, a temporal Bi-Mamba for rhythm/duration (with a lightweight training-only aligner), and a prosody predictor with gated AdaLN conditioning. To our knowledge, prior Mamba-in-speech systems (e.g., hybrids) retain inference-time attention or recurrence in duration/prosody, whereas MVC is the first to eliminate them across the entire conditioning stack. Our aim is not end-to-end speed (the diffusion decoder still dominates latency), but bounded activation memory, stable long-form synthesis, and streaming with bounded look-ahead. On LibriTTS, LJSpeech, and stress tests on VCTK/CSS10 (ES/DE/FR), MVC delivers modest but statistically significant gains in MOS/CMOS, F0~RMSE, MCD, and WER versus StyleTTS2 and Mamba–attention hybrids, while reducing conditioning parameters to 21M and improving encoder-side throughput by $1.6\times$. A runtime breakdown confirms the diffusion decoder as the primary bottleneck, whereas SSM-only conditioning lowers peak memory and enables longer inputs and larger batches in practice. Source code: \url{https://github.com/aiai-9/MVC}.
\end{abstract}

TL;DR: SSM-only TTS conditioning at inference (no attention/RNN); a gated Bi-Mamba improves long-form stability/streaming and gives ~1.6× encoder speed with modest, statistically significant quality gains.