# Sequence-to-Sequence Acoustic Modeling with Semi-Stepwise Monotonic Attention for Speech Synthesis
## 3 Proposed Methods
This is reasonable because the former utilized both textual and acoustic information. The F1 score of SSMA-PB on annotating phrase boundaries was 56.22%, which shows that without supervision those taught hidden state positions in SSMA did have a strong correlation with manually labeled phrase boundaries. ## 5 Conclusions

This paper has proposed a semi-stepwise monotonic attention (SSMA) method to improve the performance of sequence-to-sequence (Seq2 Seq) speech synthesis when phrase boundaries are not available in both training and synthesis stages. In this method, hidden states are added between adjacent phones to model the possible pauses between them. Thus, the attention to a phone at each decoding step has three options for the next decoding step, moving forward to the next phone, staying at the same phone, or jumping to a hidden state. Then, an algorithm was designed to calculate the attention weights of SSMA in a recursive way. Experimental results show that SSMA achieved better subjective performance than SMA when phrase boundaries are not available, which is quite suitable for low-resource languages that lack phrase boundaries. To improve the accuracy of SSMA-based unsupervised phrase boundary annotation and to evaluate our proposed method using the datasets of more languages will be the tasks of our future work. Conceptualization, Z.L. ; methodology, X.Z. ; implementation, X.Z. ; writing--original draft preparation, X.Z. ; writing--review and editing, Z.L. ; resources, Y.H. ; project administration, L.D. All authors have read and agreed to the published version of the manuscript. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Precision** & **Recall** & **F1 Score** \\ \hline Prediction & 32.72 & 73.61 & 45.31 \\ Annotation & 43.67 & 78.89 & 56.22 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: **Precision (%), Recall (%), and F1 score (%) of the SSMA-PB system on predicting phrase boundaries from texts and annotating phrase boundaries by forced-alignment. **Funding: This research was funded in part by the National Key R&D Program of China under Grant 2019 YFF0303001, the Major Program of National Social Science Foundation of China under Grant 15 ZDB103 and the National Nature Science Foundation of China under Grant 61871358. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: Not applicable. Conflicts of Interest: The authors declare no conflict of interest. ## References

* _[PERSON] et al. (2009)_ [PERSON]; [PERSON]; [PERSON] Statistical parametric speech synthesis. _Speech Commun._**2009**, _51_, 1039-1064. [CrossRef]
* _[PERSON] et al. (2013)_ [PERSON]; [PERSON] Text Analysis and Word Pronunciation in Text-to-speech Synthesis. In _Advances in Speech Signal Processing_; [PERSON]: New York, NY, USA, 2013; pp. 791-832. * _[PERSON] et al. (2000)_ [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON] Speech parameter generation algorithms for HMM-based speech synthesis. In Proceedings of the 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00 CH37100), Istanbul, Turkey, 5-9 June 2000; Volume 3, pp. 1315-1318. * _[PERSON] et al. (2016)_ [PERSON]; [PERSON]; [PERSON] WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications. _IEICE Trans. Inf. Syst._**2016**, _99_, 1877-1884. [CrossRef]
* _[PERSON] et al. (2016)_ [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON] WaveNet: A Generative Model for Raw Audio. In Proceedings of the 9 th ISCA Speech Synthesis Workshop (SSW9), Sunnyvale, CA, USA, 13-15 September 2016; p. 125. * _[PERSON] et al. (2017)_ [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; et al. Tacotron: Towards End-to-End Speech Synthesis. In Proceedings of the Interspeech 2017, Stockholm, Sweden, 20-24 August 2017; pp. 4006-4010. [CrossRef]
* _[PERSON] et al.