Incremental FastPitch: Chunk-based High Quality Text to Speech
Authors: name: Muyang Du; name: Chuan Liu; name: Junjie Lai
Published: 2024-01-03 14:17:35

Abstract: Parallel text-to-speech models have been widely applied for real-time speech
synthesis, and they offer more controllability and a much faster synthesis
process compared with conventional auto-regressive models. Although parallel
models have benefits in many aspects, they become naturally unfit for
incremental synthesis due to their fully parallel architecture such as
transformer. In this work, we propose Incremental FastPitch, a novel FastPitch
variant capable of incrementally producing high-quality Mel chunks by improving
the architecture with chunk-based FFT blocks, training with receptive-field
constrained chunk attention masks, and inference with fixed size past model
states. Experimental results show that our proposal can produce speech quality
comparable to the parallel FastPitch, with a significant lower latency that
allows even lower response time for real-time speech applications.