Abstract: Despite the ability to produce human-level speech for in-domain text, attention-based end-to-end text-to-speech (TTS) systems suffer from text alignment failures that increase in frequency for out-of-domain text. We show that these failures can be addressed using simple location-relative attention mechanisms that do away with content-based query/key comparisons. We compare two families of attention mechanisms: location-relative GMM-based mechanisms and additive energy-based mechanisms. We suggest simple modifications to GMM-based attention that allow it to align quickly and consistently during training, and introduce a new location-relative attention mechanism to the additive energy-based family, called Dynamic Convolution Attention (DCA). We compare the various mechanisms in terms of alignment speed and consistency during training, naturalness, and ability to generalize to long utterances, and conclude that GMM attention and DCA can generalize to very long utterances, while preserving naturalness for shorter, in-domain utterances.

Introduction: Sequence-to-sequence models that use an attention mechanism to align the input and output sequences BIBREF0, BIBREF1 are currently the predominant paradigm in end-to-end TTS. Approaches based on the seminal Tacotron system BIBREF2 have demonstrated naturalness that rivals that of human speech for certain domains BIBREF3. Despite these successes, there are sometimes complaints of a lack of robustness in the alignment procedure that leads to missing or repeating words, incomplete synthesis, or an inability to generalize to longer utterances BIBREF4, BIBREF5, BIBREF6. The original Tacotron system BIBREF2 used the content-based attention mechanism introduced in BIBREF1 to align the target text with the output spectrogram. This mechanism is purely content-based and does not exploit the monotonicity and locality properties of TTS alignment, making it one of the least stable choices. The Tacotron 2 system BIBREF3 used the improved hybrid location-sensitive mechanism from BIBREF7 that combines content-based and location-based features, allowing generalization to utterances longer than those seen during training. The hybrid mechanism still has occasional alignment issues which led a number of authors to develop attention mechanisms that directly exploit monotonicity BIBREF8, BIBREF4, BIBREF5. These monotonic alignment mechanisms have demonstrated properties like increased alignment speed during training, improved stability, enhanced naturalness, and a virtual elimination of synthesis errors. Downsides of these methods include decreased efficiency due to a reliance on recursion to marginalize over possible alignments, the necessity of training hacks to ensure learning doesn't stall or become unstable, and decreased quality when operating in a more efficient hard alignment mode during inference. Separately, some authors BIBREF9 have moved back toward the purely location-based GMM attention introduced by Graves in BIBREF0, and some have proposed stabilizing GMM attention by using softplus nonlinearities in place of the exponential function BIBREF10, BIBREF11. However, there has been no systematic comparison of these design choices. In this paper, we compare the content-based and location-sensitive mechanisms used in Tacotron 1 and 2 with a variety of simple location-relative mechanisms in terms of alignment speed and consistency, naturalness of the synthesized speech, and ability to generalize to long utterances. We show that GMM-based mechanisms are able to generalize to very long (potentially infinite-length) utterances, and we introduce simple modifications that result in improved speed and consistency of alignment during training. We also introduce a new location-relative mechanism called Dynamic Convolution Attention that modifies the hybrid location-sensitive mechanism from Tacotron 2 to be purely location-based, allowing it to generalize to very long utterances as well.

Question: What evaluation metric is used?
Answer: F1 and Weighted-F1