## 5 Materials and Methods
### Optical Model  
To compose the optical model, we adopted three architectures: two of the current state-of-the-art and one proposed for this work (presented in Section 4). In this way, we can analyze different texts' recognition methods as inputs to the Spelling Corrector. The optical model is composed of the convolutional block followed by the recurrent block; in addition, we added the Vanilla Beam Search [88] algorithm as CTC decoding (\(beam\_width=10\)), with the exception of the traditional approach, which uses Hidden Markov Model with the language model for decoding.  
The first architecture, presented by [40], is a small model that uses its own gated mechanism approach in the convolutional layers. Thus, the convolutional block is composed of: (i) 3 \(\times\) 3 convolution to extract 8 features; (ii) 2 \(\times\) 4 convolution with gated to extract 16 features; (iii) 3 \(\times\) 3 convolution with gated to extract 32 features; (iv) 2 \(\times\) 4 convolution with gated to extract 64 features; and (v) 3 \(\times\) 3 convolution to extract 128 features. In addition, Hyperbolic Tangent function (tanh) and Glorot uniform [89] is used as activator and initializer, respectively. Likewise, the recurrent block contains two BLSTM (128 hidden units per LSTM) alternated by dense layer. Figure 6 shows the overview of [PERSON] optical model architecture.  
The second architecture, presented by [15], is a big model that uses only traditional convolutional layers and several bidirectional recurrent layers; thus, its performance is equivalent to models that use multidimensional recurrent layers [54]. The convolutional block is composed of layers with  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**System**} & \multicolumn{3}{c}{**Average**} & \multicolumn{3}{c}{**Partitioning**} \\  & **Sentence Length** & \multicolumn{3}{c}{**Tokens/Sentence**} & \multicolumn{3}{c}{**Training**} & **Validation** & **Testing** & **Total** \\  & **Minimum** & **Maximum** & **Chars** & **Words** & **Training** & **Validation** & **Testing** & **Total** \\ \hline Handwritten Text Recognition & 2 & 110 & 47 & 8 & 26,342 & 3851 & 4369 & 34,562 \\ Spelling Correction & 2 & 110 & 27 & 4 & 632,582 & 70,285 & 4369 & 707,236 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Data distribution from All in One database.  
Figure 6: Bluche architecture overview.  
3 \(\times\) 3 convolution with number of features extraction following the order of 16\(n\) (16, 32, 48, 64, 80), and MaxPooling to reduce data in the first three layers. In addition, dropout is applied (rate of 0.2) in the last three layers to avoid overfitting [90], and Batch Normalization [91] is used in all convolutional layers to normalize the inputs of nonlinear activation functions. Glorot uniform [89] and Leaky Rectifier Linear Units (LeakyReLU) [92] are used as activator and initializer, respectively. The recurrent block, contains five BLSTM layers (256 hidden units per LSTM) with a dropout (rate of 0.5) per LSTM [52]. Finally, dropout (rate of 0.5) is applied before the last dense layer. Figure 7 shows the overview of Puigcerver optical model architecture.  
The training and classification phase is applied in the same way for the three optical models, which we trained each one to minimize the validation loss value of the CTC function. We used RMSprop optimizer [93] with a learning rate of 0.001 and mini-batches of 16 image samples per step. Reduce Learning Rate on Plateau (factor of 0.2) and Early Stopping mechanisms also are applied after 15 and 20 epochs, respectively, without improvement in the validation loss value.  
Furthermore, the optical models have input size of 1024 \(\times\) 128 \(\times\) 1 (Height \(\times\) Width \(\times\) Channel) and maximum length text sentence set to 128. Finally, the CTC uses the 95 printable characters from ASCII (letters, digits, punctuation marks) to decode the final text; this means that accented characters are converted to non-accented characters (normalized).