# Sequence-to-Sequence Acoustic Modeling with Semi-Stepwise Monotonic Attention for Speech Synthesis
## 3 Proposed Methods
In each test, the synthetic utterances of two systems were evaluated in random order by 11 native listeners. The listeners were asked to judge which sentence in each pair sounded more natural or there was no preference. The average preference scores are shown in Table 2. Each row in the Table 2 compares whether there are significant differences in different systems. The percentage of the better system is shown in bold format. From this table, we can see that the SSMA-PB system outperformed the SMA-PB system significantly with \(p<0.001\). This result indicates that using SSMA helped Tacotron2 to synthesize speech with better naturalness when phrase boundaries were not given in both training and synthesis stages. On the other hand, the subjective performance of SSMA-PB was still not as good as SMA+PB which utilized manually labeled phrase boundaries (\(p<0.001\)). ### Discussions

To explore the interpretability of our proposed SSMA method, two experiments were conducted to evaluate the consistency between the pause positions derived from SSMA-based alignment and manually labeled phrase boundaries. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **MCD (dB)** & **RMSE (Hz)** & **CORR** & **UV (\%)** \\ \hline SMA-PB & 3.73 & 46.49 & 0.83 & 7.77 \\ SSMA-PB & 3.65 & 46.13 & 0.83 & 7.78 \\ SMA+PB & 3.53 & 41.95 & 0.88 & 7.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Objective performance of SMA-PB, SSMA-PB, and SMA+PB on predicting the acoustic features of test sentences, where **MCD**, **RMSE**, **CORR**, and **UV** denote the mel-cepstral distortion, F0 root mean square error, F0 correlation, and UV error percentage, respectively. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**SMA-PB** & **SSMA-PB** & **SMA+PB** & **N/P** & \(p\) \\ \hline
21.36 & **54.55** & - & 24.09 & \textless{}0.001 \\ - & 26.36 & **45.91** & 27.73 & \textless{}0.001 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Subjective preference scores (%) among SMA-PB, SSMA-PB, and SMA+PB systems on the test set, where N/P denotes \"No Preference\" and \(p\) means the \(p\)-value of \(t\)-test between two systems. #### 4.4.1 Predicting Phrase Boundaries from Texts

This experiment evaluated the accuracy of predicting phrase boundaries from texts using the pause positions determined by SSMA at the inference stage. The sentences in the test set were synthesized by SSMA-PB with hard mode. If the hidden state between two adjacent phones was assigned more than one frame at the decoding time, a phrase boundary was predicted between these two phones. The hidden states adjacent to silence phones were not considered. Evaluation metrics included the precision, recall, and F1 score of predicting phrase boundaries. The results are shown in the first row of Table 3. We can see that SSMA-PB achieved a recall of 73.61%, which means that most true phrase boundaries can be found by SSMA-based decoding. However, its precision was much lower. One possible reason is that there were short pauses determined by SSMA, which may not correspond to true phrase boundaries. It should be noticed that SSMA-PB predicted phrase boundaries in an unsupervised way, i.e., no phrase boundary annotations were utilized at the training stage. #### 4.4.2 Annotation Phrase Boundaries by Forced Alignment

This experiment evaluated the accuracy of annotating phrase boundaries by SSMA-based forced alignment when both texts and recordings were given. In this case, the decoder was conducted in a teacher-forcing way, which means that the true history of the mel-spectrogram was used as input at each decoding step. After alignment paths were calculated, the phrase boundaries were annotated following the conditions used in previous experiment. The results of phrase boundary annotation are shown in the second row of Table 3, where the same metrics used in previous experiment were employed. We can see that SSMA-based phrase boundary annotation achieved higher precision, recall, and F1 score than SSMA-based phrase boundary prediction.