# Graphene-to-Phoneme Conversion with Convolutional Neural Networks
## 5 Discussion  
In this section, we discuss the results of the previous section and analyze the connection between PER values and word length, furthermore the position of the error within the word. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Data** & **Method** & **PER (\%)** & **WER (\%)** & **Time (s)** & \
\begin{tabular}{c} **Number** \\ **of Epochs** \\ \end{tabular} &
\begin{tabular}{c} **Model** \\ **Size** \\ \end{tabular} \\ \hline
1 & Encoder-Decoder LSTM & 5.68 & 28.44 & 467.73 & 185 & 12.7 M \\
1A & Encoder-Decoder LSTM with attention layer & 5.23 & 28.36 & 688.9 & 136 & 13.9 M \\
2 & Encoder-Decoder Bi-LSTM & 5.26 & 27.07 & 858.93 & 177 & 33.8 M \\
2A & Encoder-Decoder Bi-LSTM with attention layer & 4.86 & 25.67 & 1045.5 & 114 & 35.3 M \\
3 & Encoder CNN, decoder Bi-LSTM & 5.17 & 26.82 & 518.3 & 115 & 13.1 M \\
4 & End-to-end CNN (with res. connections) & 5.84 & 29.74 & 176.1 & 142 & 7.62 M \\
5 & Encoder CNN with res. connections, decoder Bi-LSTM & 4.81 & 25.13 & 573.5 & 147 & 14.5 M \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Results on the CMUDict dataset. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Data** & **Method** & **PER (\%)** & **WER (\%)** \\ \hline \multirow{8}{*}{NetTalk} & Joint sequence model [3] & 8.26 & 33.67 \\  & Bi-LSTM [16] & 7.38 & 30.77 \\  & Encoder-decoder with global attention [32] & 7.14 & 29.20 \\  & **Encoder CNN with residual connections, decoder Bi-LSTM** & **5.69** & **30.10** \\ \hline \multirow{8}{*}{CMUDict} & LSTM with Full-delay [12] & 9.11 & 30.1 \\  & Joint sequence model [3] & 5.88 & 24.53 \\  & Encoder-decoder LSTM [16] & 7.63 & 28.61 \\  & Bi-LSTM + Alignment [16] & 5.45 & 23.55 \\  & Combination of sequitur G2P and seq2 seq-attention and multitask learning [26] & 5.76 & 24.88 \\  & Ensemble of 5 [Encoder-decoder + global attention] models [32] & 4.69 & 20.24 \\  & Encoder-decoder with global attention [32] & 5.04 & 21.69 \\  & Joint multi-gram + CRF [8] & 5.5 & 23.4 \\  & Joint n-gram model [4] & 7.0 & 28.5 \\  & Joint maximum entropy (ME) n-gram model [5] & 5.9 & 24.7 \\  & Encoder-Decoder GRU [9] & 5.8 & 28.7 \\  & **Encoder CNN with residual con., decoder Bi-LSTM (fifth model)** & **4.81** & **25.13** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of best previous results of G2P models with our fifth model (encoder is a CNN with residual connections, Bi-LSTM decoder) on CMUDict and NetTalk. We categorize the word length into 3 classes: short (shorter than 6 characters), medium (between 6 and 10 characters), long (more than 10 characters). According to this categorization, there were 4306 short, 5993 medium and 1028 long words in the CMUDict dataset. In this analysis, we ignored approximately 600 words that have multiple pronunciation variants in the reference data. The results of this comparison are presented in Figure 6a. For short words, all models show similar PERs; for medium length words, except the end-to-end CNN model (fourth model), the other models resulted in similar error; for long words, encoder CNN with residual connection, decoder Bi-LSTM (fourth model) and encoder CNN, decoder Bi-LSTM (third model) got similar minimum errors. The fourth model showed the highest error in both medium and long length words. According to Figure 6a, the advantage of Bi-LSTM-based models is clearly shown for learning long sequences. Moreover, errors occurring in the first half of the pronunciation (in the reference) increases the probability of predicting incorrect phonemes in the second half. Still, a correctly predicted first half cannot guarantee a correctly predicted second half. In our experiments, convolutional architectures also performed well on short and on long-range dependencies. Our intuition is that the residual connections enable the network to consider features learned by lower and higher layers--which represents shorter and longer dependencies.