Utilizing POS -Driven Pitch Contour Analysis for Enhanced T amil Text-to-Speech Synthesis (2024)

Abstract: A novel approach to text-to-speech synthesis that integrates pitch contour labels derived from the highest occurrence analysis for each Part-of-Speech (POS) tag. Using the Stanford POS Tagger, grammatical tags are assigned to words, and the most frequently occurring pitch contour labels associated with these tags are analyzed, focusing on both unigram and bigram statistics. The primary goal is to identify the pitch contour for each POS tag based on its frequency of occurrence. These pitch contour labels are incorporated into the output of the synthesized waveform using the TD-PSOLA (Time Domain Pitch Synchronous Overlap and Add) signal processing algorithm. The resulting waveform is evaluated using Mean Opinion Scores (MOS), demonstrating significant enhancements in quality and producing a prosodically rich synthetic speech.

## introduction text-to-speech (tts) systems convert written text into natural-sounding speech, with prosody playing a vital role in achieving high-quality synthesis. prosody encompasses pitch, rhythm, and intensity, with pitch contour being particularly important for conveying meaning and emotion in spoken language. natural human communication sees pitch contour influenced by various factors, including pitch accents, phrase boundaries, and intonation patterns @xcite . while significant advancements have been made in integrating prosodic features into tts systems across various languages, tamil-a dravidian language known for its complex tonal and prosodic structures-remains underexplored. existing research has investigated methods for enhancing prosody in tts, such as statistical parametric speech synthesis (spss), which offers a range of techniques to improve spoken output @xcite . deep learning models like tacotron and wavenet have made substantial strides in generating natural-sounding speech by learning direct mappings from text to audio waveforms @xcite . however, there is a need for approaches that effectively combine linguistic elements-such as part-of-speech (pos) tags-with prosodic features to enhance the naturalness and intelligibility of synthesized speech. in this regard, techniques like linear prediction (lp) and time-domain pitch synchronous overlap and add (td-psola) have been utilized to enhance naturalness and expressiveness @xcite @xcite . the objective of the current work is to enhance a synthesized speech signal, by analysing pitch contour labels associated with various pos tags. utilizing the stanford pos tagger, grammatical tags are assigned to words in tamil utterances, and the most frequently occurring pitch contours for each tag are analysed through unigram and bigram statistics. this analysis enables the integration of the most common pitch contours into synthesized speech. the identified pitch contours will be incorporated to the synthesized speech signal using the td-psola technique, resulting in more natural and expressive tamil speech synthesis. to evaluate the effectiveness of this approach, a mean opinion score (mos) is obtained. preliminary results indicate that this method achieves a higher mos score compared to the output from the hmm-based tts system, demonstrating its potential to significantly improve the naturalness of synthesized tamil speech. the paper is organized as follows: section 2 outlines the methodology employed for pos tagging and pitch contour analysis. section 3 presents the proposed approach, highlighting the integration of pitch contour labels based on pos tags. section 4 discusses the results and analysis. finally, section 5 concludes the paper and outlines directions for future work. the objective of this study is to identify the predominant pitch contour labels associated with different part-of-speech (pos) tags in tamil language utterances using the mile corpus and validate these findings through a testing phase. ## prosody modification in synthesized speech given a text, corresponding to a sentence, using unigram and bigram statistics, for each of the words, a suitable pitch contour is assigned. the identified pitch contour labels are incorporated into the synthesized waveform generated by the hidden markov model (hmm)-based tts system @xcite , known for its small footprint and reduced redundancy. the output waveform from the hts system typically has a neutral tone. to enhance expressiveness, pitch contour modifications are applied using the time-domain pitch-synchronous overlap-add (td-psola) technique. this method allows for accurate pitch adjustments while preserving the natural quality of the original audio signal, ensuring the synthesized speech retains both clarity and emotional nuance. to directly modify the prosody of speech without relying on modeling, a signal processing algorithm known as time-domain pitch-synchronous overlap and add (td-psola) is employed @xcite . since it operates pitchsynchronously, the instants of excitation, or glottal closure instants (gcis), are derived using a phasebased algorithm (pd) @xcite . the speech signal is segmented using a hanning window, with segments centered around the gcis. to modify both the pitch contour, the algorithm selects, overlaps, and adds the segments aligned with the new excitations, preserving naturalness in the synthesized speech. this approach allows for precise pitch modifications without altering the duration of the original signal, resulting in a synthesized output that retains the natural characteristics of the speech while achieving the desired pitch contour adjustments. changes in pitch contours significantly influence the expression of emotions in the synthesized waveform. for instance, a rising pitch contour may convey excitement or enthusiasm, while a falling contour can indicate sadness or calmness. by accurately adjusting the pitch according to these emotional cues, the synthesized speech can better reflect human-like emotional expressions. finally, the generation of the modified waveform aims to produce output with natural prosody. by integrating pitch adjustments that align with the prosodic features of human speech, the synthesized audio achieves a more fluid and realistic sound, enhancing listener engagement and comprehension. the figure 2 illustrates the comparison between the pitch contour of the synthesized speech signal and the pitch contour processed using the td-psola-based signal processing algorithm. the effectiveness of td-psola lies in its ability to operate at the signal level, minimizing distortion and ensuring a high-quality synthesized output. ## results and analysis the performance of the proposed method was evaluated using the mean opinion score (mos) as a subjective measure to assess the effectiveness of pitch contour fitting for each word segment, incor-porating the contours into the full utterance. a total of 10 audio samples were synthesized, and mos scores were obtained from 10 listeners, resulting in an average score of 5. the mos score obtained from hts is 3.58. when comparing the synthesized output with pitch contour modifications applied through the td-psola technique to the baseline synthesized output from the hts system, the modified output achieved an mos score of 4.1 out of 5. this score indicates a significant improvement in prosody, showcasing the effectiveness of pitch contour fitting based on part-of-speech (pos) tag analysis. to extend the scope of this analysis, the pitch contour modifications were also incorporated into synthesized waveforms generated by state-of-theart neural tts models, namely tacotron and fast-speech2. tacotron, known for its sequence-tosequence modeling capability, achieved an mos score of 4.35 when augmented with the pitch contour modifications. fastspeech2, a parallel tts model designed for faster synthesis, achieved an mos score of 4.3 with the same enhancements. these results indicate that the integration of pitch contour fitting is effective across different tts frameworks, improving prosody and naturalness consistently. the enhanced scores across all models highlight the ability of the modified synthesis process to convey emotional expressiveness and naturalness, bridging the gap between machine-generated and human-like speech. by integrating statistical analyses of pitch contours into the synthesis process, the results show a clear improvement in the prosody of synthesized speech, providing a more engaging and natural listening experience. these findings underscore the potential of incorporating advanced prosodic features into tts systems for applications requiring high-quality, expressive speech synthesis. ## conclusion in this paper, a novel approach to tamil text-tospeech synthesis that integrates pitch contour based on analysis of part-of-speech (pos) tagging is proposed. our results demonstrate significant improvements in the naturalness and emotional expressiveness of synthesized speech, as indicated by the mean opinion scores (mos) obtained from listener evaluations. by leveraging the td-psola technique, we successfully incorporated pitch contour modifications, leading to a more engaging auditory experience that reflects the nuances of human communication. while the model effectively captures pitch variations, further refinements are necessary to encompass a broader range of emotional expressions and linguistic features.