# Graphene-to-Phoneme Conversion with Convolutional Neural Networks
## 3 Convolutional Neural Networks for Grapheme to Phoneme Conversion
### Details of the Bidirectional Decoder  
The details of the bidirectional decoder, which was used in Model 2, are presented in this section. Given an input sequence \(x=(x_{1},x_{2},\dots,\,x_{N})\), the LSTM network computes the hidden vector sequence \(h=(h_{1},h_{2},\dots,h_{N})\) and output vector sequence \(y=(y_{1},y_{2},\dots,y_{N})\). Initially, one-hot character vectors for graphemes and phonemes sequences were created. Character vocabularies, which contain all the elements that are present in the input and output  
Figure 3: G2P conversion based on (**a**) convolutional neural network with residual connections (fourth model); and (**b**) encoder convolutional neural network with residual connections and decoder Bi-LSTM (fifth model). **f, d, s** are the number of the filters, length of the filters and stride, respectively. data, are separately calculated. In other words, neither a grapheme vector in the output vocabulary, nor a phoneme vector in the input vocabulary, was used. These were the inputs to the encoder and the decoder. Padding was applied to make all input and output sequences to have the same length, which was set to 22. This number (22) was chosen based on the maximum length in the training database. For G2P, \(x=(x_{1},x_{2},\dots,\ x_{N})\) are one-hot character vectors of grapheme sequences; \(y=(y_{1},y_{2},\dots,y_{N})\) are one-hot character vectors of phoneme sequences. In the proposed Model 2, Bi-LSTM was used as an encoder, and it consists of two LSTMs: one that processes the sequence from left-to-right (forward encoder), and one that does it in reverse (backward encoder). This was applied to learn the semantic representation of the input sequences in both directions. One LSTM looks at the sequence from left-to-right (forward encoder), and so reads an input sequence in left-to-right order; and another LSTM looks at it in reverse (backward encoder), and so reads an input sequence in a right-to-left order. At each of the time steps, the forward hidden sequence \(\vec{h}\) and the backward hidden sequence \(\overleftarrow{h}\) are iterated by the following equations [49]:  
\[\overrightarrow{h_{t}} =\mathcal{H}(W_{ \begin{subarray}{c}\to\\ x\end{subarray} }x_{t}+W_{ \begin{subarray}{c}\to\\ h\end{subarray} }\overrightarrow{h}_{t-1}+b_{ \begin{subarray}{c}\to\\ h\end{subarray} }), \tag{1}\] \[\overleftarrow{h_{t}} =\mathcal{H}(W_{ \begin{subarray}{c}\to\\ x\end{subarray} }x_{t}+W_{ \begin{subarray}{c}\to\\ h\end{subarray} }\overleftarrow{h}_{t+1}+b_{ \begin{subarray}{c}\to\\ h\end{subarray} }), \tag{2}\]  
In Equation (1), the forward layer is iterated from \(t=1\) to \(N\); in Equation (2), the backward layer is iterated from \(t=N\) to \(1\); \(\mathcal{H}\) is an element-wise sigmoid function. As the next step, the hidden states of these two LSTMs were concatenated to form an annotation sequence \(h=\{h_{1},h_{2},\ \dots,h_{N}\}\), where \(h_{t}=\left(\overrightarrow{h_{t}},\overleftarrow{h_{t}}\right)\) encodes information about the \(t\) - th grapheme with respect to all the other surrounding graphemes in the input. \(W_{ \begin{subarray}{c}\to\\ x\end{subarray} }\), \(W_{ \begin{subarray}{c}\to\\ x\end{subarray} }\), \(W_{ \begin{subarray}{c}\to\\ h\end{subarray} }\) and \(W_{ \begin{subarray}{c}\to\\ h\end{subarray} }\) are weight matrixes; \(b_{ \begin{subarray}{c}\to\\ h\end{subarray} }\), \(b_{ \begin{subarray}{c}\to\\ h\end{subarray} }\) denotes the bias vectors. Generally, in all parameters, the arrows pointing left to right and right to left refer to the forward and backward layers, respectively. The forward LSTM unrolls the sequences until it reaches the end of sequence for that input. The backward LSTM unrolls the sequences until it reaches the start of the sequence. For the decoder, we used bidirectional LSTM. These LSTMs can be called forward and backward decoders, and described as \(\overrightarrow{d}\), \(\overleftarrow{d}\). After concatenating the forward and backward encoder LSTMs, the backward decoder performs decoding in a right-to-left way.