# Graphene-to-Phoneme Conversion with Convolutional Neural Networks
## 4 Experiments
### Training  
For the CMUDict experiments, in all models, the size of the input layers is equal to the input: [length of the longest input (22) \(\times\) number of graphemes (27)] and the size of the output layers is equal to the output: [length of the longest output (22) \(\times\) number of phonemes (41)].  
To transform graphemes and phonemes for neural networks, we convert inputs into 27-dimensional and outputs to 41-dimensional one-hot vector representations. For example, the phoneme sequences of the word 'ARREST' is 'ER EH S T'; the input and output vectors of the grapheme and phoneme sequences are as below:  
Figure 4: The architecture of the proposed bidirectional decoder model for G2P task.  
Input vector of 'ARREST':  
\[\left[\begin{array}{cccccccccccc}&A&B&C&E&\cdots&R&\cdots&S&T&\cdots\\ A&1&0&0&0&0&0&0&0&0&0\\ R&0&0&0&0&0&1&0&0&0&0\\ R&0&0&0&0&0&1&0&0&0&0\\ E&0&0&1&0&0&0&0&0&0\\ S&0&0&0&0&0&0&1&0&0\\ T&0&0&0&0&0&0&0&1&0\\ \cdots&0&0&0&\cdots&0&0&0&\cdots&\cdots&0\end{array}\right]\]  
Output vector of 'ER EH S T':  
\[\left[\begin{array}{cccccccc}&/EP&\cdots&EH&ER&EP&\cdots&S&SH&T&\cdots\\ EP&0&0&0&1&0&0&0&0&0\\ ER&0&0&1&0&0&0&0&0&0\\ EH&0&0&1&0&0&0&0&0&0\\ S&0&0&0&0&0&1&0&0\\ T&0&0&0&0&0&0&0&1&0\\ /EP&1&0&0&0&0&0&0&0&0\\ \cdots&0&0&0&\cdots&0&0&0&\cdots&\cdots&0\end{array}\right]\]  
In the case of LSTMs, we applied the Adam optimization algorithm [50] with a starting learning rate of 0.001, and with baseline values of \(\beta_{1}\), \(\beta_{2}\) and \(\varepsilon\) (0.9, 0.999 and \(1\times 10^{-8}\), respectively). For batch size, 128 was chosen. Weights were saved when the PER on the validation dataset achieved a lower value than before. When the PER did not decrease further for 100 epochs, the best model was chosen, and it was trained with stochastic gradient descent (SGD) further. In the case of the first, second and third models for SGD, we used 0.005 as the learning rate, and 0.8 for momentum. For the fourth (convolutional with residual connections) model, 0.05 (learning rate) and 0.8 (momentum) were applied, and it was trained for 142 when early stopping was called. In the fifth model, 0.5 (learning rate) of SGD and 0.8 (momentum) was set, and when PER stopped improving for about 50 epochs, the learning rate was multiplied by 4/5. The numbers of epochs for this model reached 147 and 135 for CMUDict and NetTalk, respectively.  
In all proposed models, the patience of early stopping was set to 50 in the Adam optimizer and 30 in the SGD optimizer. For NetTalk experiments, the sizes of the input and output layers are as follows: input--((length of the longest input (19) \(\times\) number of graphemes (26))); and output--[length of the longest output (19) \(\times\) number of phonemes (52)].  
We converted inputs to 26-dimensional and outputs to 52-dimensional one-hot vector representations as in case of CMUDict. The same model structure was used as with the CMUDict experiments.  
Moreover, the implementation of a single convolutional layer on input data is presented in Figure 5. The input is a one-hot vector of 'ARREST'; 64 filters of (input length) \(\times 3\) are applied to the input. In other words, the input is convolved with 64 feature maps, which produce the output of the convolutional layer. Zero padding was used to ensure that the output of the convolution layer has the same dimension as the input. During training, the filter weights are optimized to produce lower loss values.  
During inference, prediction of the graphemes sequence is decoded until </EP>, and the length of input and output are not considered.