## 4 Results and Discussion
### Subjective Evaluations  
In addition to the objective measurements, subjective listening tests were performed to evaluate the perceptual quality of our enhanced speech samples in terms of intelligibility and speech quality. The most frequently subjective tests were conducted. For both tests, each participant evaluated the voice quality of twenty-two sentences (11 different test utterances spoken by 2 different pairs of speakers) from each of the previous types of voice. All tests were carried out under the same conditions and based on the same principle. 1. MOS test (Mean Opinion Score): used to evaluate the speech quality and intelligibility of the resynthesized voice. In this experiment, a group of fifteen auditors (five males and ten females) listened to a set of sample utterances and judged them independently, one by one, according to a rating scale of perceived quality. This scale ranged from (1) for the poorest quality to (5) for excellent quality, ((2) poor, (3) average, and (4) good \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**MODELS** & **CD Value** & **PESQ Score** & **STOI Score** & **segSNR Value** \\ \hline ES-SRC & 9.408 & 2.407 & 0.606 & 2.981 \\ JD-GMM & 8.795 & 2.102 & 0.518 & 9.706 \\ DNN & 8.257 & 2.570 & 0.544 & 10.099 \\ LSTM & 8.009 & 2.808 & 0.624 & 10.915 \\ BiLSTM & 7.311 & 2.914 & 0.641 & 11.943 \\ Seq2 Seq & **6.836** & **2.994** & **0.733** & **12.854** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Results of the objective assessment of the reference and proposed methods on the PC speaker esophageal corpus test set. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**MODELS** & **CD Value** & **PESQ Score** & **STOI Score** & **segSNR Value** \\ \hline ES-SRC & 9.153 & 2.381 & 0.597 & 2.999 \\ JD-GMM & 8.861 & 2.179 & 0.513 & 9.741 \\ DNN & 8.405 & 2.619 & 0.581 & 10.305 \\ LSTM & 7.910 & 2.805 & 0.624 & 11.083 \\ BiLSTM & 7.127 & 2.903 & 0.633 & 11.977 \\ Seq2 Seq & **6.605** & **3.002** & **0.775** & **12.901** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results of the objective assessment of the reference and proposed methods on the MH speaker esophageal corpus test set. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**VARIANTS** & **CD Value** & **PESQ Score** & **STOI Score** & **segSNR Value** \\ \hline BiLSTM+Attention & **6.994** & **2.933** & **0.718** & **12.407** \\ BiLSTM & 7.311 & 2.914 & 0.641 & 11.943 \\ LSTM+Attention & **7.826** & **2.871** & **0.639** & **11.866** \\ LSTM & 8.009 & 2.808 & 0.624 & 10.915 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance comparison between different variants on the PC speaker esophageal corpus test set. quality)). The average score awarded, therefore, constituted the MOS, which indicated the intelligibility and the quality of the enhanced speech. For these tests, we conducted an opinion test for intelligibility and another opinion test for naturalness. Five sets of comparative experiments were evaluated by fifteen auditors. * ES-SRC: source esophageal speech;
* JD-GMM: conversion system based on a joint density Gaussian mixture model;
* DNN: conversion system based on feed-forward DNN model;
* BiLSTM: conversion system based on Bidirectional LSTM model;
* Seq2 Seq: conversion system based on sequence to sequence with attention mechanism model. Figures 5 and 6 show the MOS values on intelligibility and speech quality using different conversion methods. From both figures, we can see that the method based on GMM has the lower performance when compared with all the other methods, and this method has a high refusal rate by the laryngectomees. The LSTM method achieved almost similar intelligibility and naturalness when compared with the BiLSTM method, which in turn outperformed the DNN-based method, with a slight improvement. On the other hand, we can clearly notice that the converted speech using our method achieves the highest MOS in both tests.