  We propose a novel phrase break prediction method that combines implicit
features extracted from a pre-trained large language model, a.k.a BERT, and
explicit features extracted from BiLSTM with linguistic features. In
conventional BiLSTM based methods, word representations and/or sentence
representations are used as independent components. The proposed method takes
account of both representations to extract the latent semantics, which cannot
be captured by previous methods. The objective evaluation results show that the
proposed method obtains an absolute improvement of 3.2 points for the F1 score
compared with BiLSTM-based conventional methods using linguistic features.
Moreover, the perceptual listening test results verify that a TTS system that
applied our proposed method achieved a mean opinion score of 4.39 in prosody
naturalness, which is highly competitive with the score of 4.37 for synthesized
speech with ground-truth phrase breaks.
