Introduction
Since early times of computer-based speech synthesis research, voice quality (the perceived timbre of speech) analysis/modification has attracted interest of researchers BIBREF0. The topic of voice quality analysis finds application in various areas of speech processing such as high-quality parametric speech synthesis, expressive/emotional speech synthesis, speaker identification, emotion recognition, prosody analysis, speech therapy. Due to availability of reviews such as BIBREF1 and space limitations, a review of voice quality analysis methods will not be presented here.
For voice quality analysis of speech corpora, it is common practice to estimate spectral parameters directly from speech signals such as relative harmonic amplitudes, or Harmonic to Noise Ratio (HNR). Although the voice quality variations are mainly considered to be controlled by the glottal source, glottal source estimation is considered to be problematic and hence avoided in the parameter estimation procedures for processing large speech corpora. In this work, we follow the not so common path and study the differences present in the glottal source signal parameters estimated via an automatic algorithm when a given speaker produces different voice qualities. Based on a parametric analysis of these latter (Section SECREF2), we further investigate the use of the information extracted from a large corpus, for voice quality modification of other speech databases in a HMM-based speech synthesizer (Section SECREF3).
Excitation-based Voice quality analysis
The goal of this part is to highlight the differences present in the excitation when a given speaker produces different voice qualities. The De7 database used for this study was designed by Marc Schroeder as one of the first attempts of creating diphone databases for expressive speech synthesis BIBREF2. The database contains three voice qualities (modal, soft and loud) uttered by a German female speaker, with about 50 minutes of speech available for each voice quality. In Section SECREF1, the glottal flow estimation method and glottal flow parametrization used in this work are briefly presented. The harmonicity of speech is studied via the maximum voiced frequency in Section SECREF3. As an important perceptual charactersitic, spectral tilt is analyzed in Section SECREF4. Section SECREF6 compares the so-called eigenresiduals BIBREF3 of the different voice qualities. Finally Section SECREF8 quantifies the separability between the three voice qualities for the extracted excitation features.
Excitation-based Voice quality analysis ::: Glottal source
We recently showed that complex cepstrum can be efficiently used for glottal flow estimation BIBREF4. This method aims at separating the minimum and maximum-phase components of the speech signal. Indeed it has been shown previously BIBREF5 that speech is a mixed-phase signal where the maximum-phase (i.e anti-causal) contribution corresponds to the glottal open phase, while the minimum-phase component is related to the vocal tract transmittance (assuming an abrupt glottal return phase). Isolating the maximum-phase component of speech then provides a reliable estimation of the glottal source, which can be achieved by the complex cepstrum. The glottal flow open phase is then parametrized by three features: the glottal formant frequency ($F_g$), the Normalized Amplitude Quotient ($NAQ$) and the Quasi-Open Quotient ($QOQ$).
The glottal formant was tracked using the method described in BIBREF6. Figure FIGREF2(a) shows the histograms of $Fg/F_0$ for the three voice qualities. Significant differences between the distributions are observed. Among others it turns out that a louder (softer) voice results in the production of a higher (lower) glottal formant frequency. Another observation that can be drawn from this figure is the presence of two modes for the modal and loud voices. This may be explained by the fact that the estimated glottal source sometimes comprises a ripple both in the time and frequency domains, which in turn may have two possible causes: an incomplete separation between $Fg$ and the first formant $F_1$ BIBREF6, and/or a non-linear interaction between the vocal tract and the glottis BIBREF7. This ripple may therefore affect the detection of the glottal formant frequency and in this way explain the parasitical peak in the $Fg/F_0$ histogram for the modal and loud voices.
In previous works BIBREF8, BIBREF9, Alku et al. proposed the Normalized Amplitude Quotient and the Quasi-Open Quotient as two efficient time-domain parameters characterizing respectively the closing and open phase of the glottal flow. These parameters are extracted using the Aparat toolkit BIBREF10 from the glottal source estimated here by the complex cepstrum . Figures FIGREF2(b) and FIGREF2(c) display the histograms of these two features for the three voice qualities. Notable differences between histograms may be observed.
Excitation-based Voice quality analysis ::: Maximum Voiced Frequency
Some approaches, such as the Harmonic plus Noise Model (HNM ,BIBREF11), consider that the speech signal can be modeled by a non-periodic component beyond a given frequency. In the case of HNM, this maximum voiced frequency ($F_m$) demarcates the boundary between two distinct spectral bands, where respectively an harmonic and a stochastic modeling are supposed to hold. The higher the $F_m$, the stronger the harmonicity, and consequently the weaker the presence of noise in speech. In this paper, $F_m$ was estimated using the algorithm described in BIBREF11. Figure FIGREF2(d) displays the histograms of $F_m$ for the three voice qualities. It can be observed that, in general, the soft voice has a low $F_m$ (as a result of its breathy nature) and that the stronger the vocal effort, the more harmonic the speech signal, and consequently the higher $F_m$.
Excitation-based Voice quality analysis ::: Spectral Tilt
Spectral tilt of speech is known to play an important role in the perception of a voice quality BIBREF12. To capture this crucial feature, an averaged spectrum is obtained on the whole corpus by a process independent of the prosody and the vocal tract variations. For this, voiced speech frames are extracted by applying a two pitch period-long Hanning windowing centered on the current Glottal Closure Instant (GCI). GCI locations are determined using the technique described in BIBREF13. These frames are then resampled on a fixed number of points (corresponding to two mean pitch periods) and normalized in energy. The averaged spectrum is finally achieved by averaging the spectra of the normalized speech frames. The averaged amplitude spectrum then contains a mix of the average glottal and vocal tract contributions. The averaged spectrum for the three voice qualities is exhibited in Figure FIGREF5. Since these spectra were computed for the same speaker, it is reasonable to think that the main difference between them is due to the spectral tilt regarding the produced voice quality. Among others it can be noticed from Figure FIGREF5 that the stronger the vocal effort, the richer the spectral content in the [1kHz-5kHz] band.
Excitation-based Voice quality analysis ::: Eigenresiduals
In BIBREF3 we proposed to model the residual signal by decomposing speaker-dependent pitch-synchronous residual frames on an orthonormal basis. It was also shown that the first so-obtained eigenvector (called eigenresidual) can be efficiently used in parametric speech synthesis. As eigenresiduals are employed in our voice quality modification application in Section SECREF3, Figure FIGREF7 displays the differences present in this signal depending on the produced voice quality. It can be noticed that the conclusions we drew in Section SECREF1 about the glottal open phase are corroborated. It is indeed observed that the stronger the vocal effort, the faster the response of the eigenresidual open phase.
Excitation-based Voice quality analysis ::: Separability between Distributions
Important differences in the distributions of the features have been presented in the previous subsections (which are in line with the conclusions presented in various studies BIBREF0, BIBREF8, BIBREF12). In this section, we quantify how these differences between voice qualities are significative. For this, the Kullback-Leibler (KL) divergence BIBREF14 is known to measure the separability between two discrete density functions $A$ and $B$. But since this measure is non-symmetric (and consequently is not a true distance), its symmetrised version, called Jensen-Shannon divergence BIBREF14, is often prefered. It consists of a sum of two KL measures:
where $M$ is the average of the two distributions ($M=0.5*(A+B)$). Table TABREF10 shows the results for the four features we previously presented. Among others it can be noticed that the loud and soft voices are highly separable, while the loud type is closer to the modal voice than the soft one. It is also seen that $F_g$ and $NAQ$ are highly informative for voice quality labeling.
Voice quality modification
In a previous work BIBREF3, we proposed a Deterministic plus Stochastic Model (DSM) of the residual signal. In this approach, the excitation is divided into two distinct spectral bands delimited by the maximum voiced frequency $F_m$. The deterministic part concerns the low-frequency contents and is modeled by the first eigenresidual as explained in Section SECREF6. As for the stochastic component, it is a high-pass filtered noise similarly to what is used in the HNM BIBREF11. The residual signal is then passed through a LPC-like filter to obtain the synthetic speech. This section aims at applying voice quality modification as a post-process to HMM-based speech synthesis BIBREF15 using the DSM of the residual signal. More precisely, a HMM-based synthesizer is trained on a corpus of modal voice for a given speaker. The goal is then to transform the synthetic voice so that it is perceived as soft or loud, while avoiding a degradation of quality in the produced speech.
Since no dataset of expressive voice is available for the considered speaker, modifications are extrapolated from the prototypes described for speaker De7 in Section SECREF2, assuming that other speakers modify their voice quality in the same way. Three main transformations are here considered:
The eigenresiduals presented in Section SECREF6 are used for the deterministic part of the DSM. These waveforms implicitly convey the modifications of glottal open phase that were underlined in Section SECREF1.
The maximum voiced frequency $F_m$ is fixed for a given voice quality according to Section SECREF3 by taking its mean value: 4600 Hz for the loud, 3990 Hz for the modal (confirming the 4 kHz we used in BIBREF3), and 2460 Hz for the soft voice.
The spectral tilt is modified using the inverse of the process described in Section SECREF4. For this, the averaged spectrum of the voiced segments is transformed, in the pitch-normalized domain, by a filter expressed as a ratio between auto-regressive modelings of the source and target voice qualities (cf Fig.FIGREF5). Residual frames are then resampled to the target pitch at synthesis time. This latter transformation is then pitch-dependent.
To evaluate the technique, a subjective test was submitted to 10 people. The test consisted of 27 sentences generated by our system for three speakers (two males and one female). One third of these sentences were converted to a softer voice, and one third to a louder one. For each sentence, participants were asked to assess the vocal effort they perceive (0 = very soft, 100 = very tensed), and to give a MOS score. Results are displayed in Table TABREF14 with their 95% confidence intervals. Interestingly it can be noticed that voice quality modifications are perceived as expected while the overall quality is not significantly altered (although listeners have a natural tendency to prefer softer voices).
Conclusion
In this study we show that a glottal flow estimation algorithm BIBREF4 can be effectively used for voice quality analysis on large speech corpora where most of glottal flow estimation literature are based on tests with sustained vowels. We study the variations in parameters for different voice qualities and conclude that the two glottal flow parameters $F_g$ and $NAQ$ are highly informative for voice quality labeling. We further show that the information extracted from one speech database can be applied to other speech databases for voice quality modification and the quality achieved in a speech synthesis application is fairly high.
Acknowledgments
Thomas Drugman is supported by the “Fonds National de la Recherche Scientifique” (FNRS). The authors would like to thank M. Schroeder for the De7 database, as well as Y. Stylianou for providing the algorithm extracting $F_m$.

Question: What large corpus is used for experiments?
Answer: The De7 database