# End-to-End Neural Optical Music Recognition of Monophonic Scores
## 4 Neural End-to-end Approach for Optical Music Recognition
### Implementation Details  
The objective of this work is not to seek the best neural model for this task, but to study the feasibility of this framework. Thus, a single neural model is proposed that, by means of informal testing, has verified its goodness for the task. The details concerning the configuration of the neural model are given in Table 3. As observed, input variable-width single-channel images (grayscale) are rescaled at a fixed height of 128 pixels,  
Figure 4: Graphical scheme of the end-to-end neural approach considered. without modifying their aspect ratio. This input is processed through a convolutional block inspired by a VGG network, a typical model in computer vision tasks [49]: four convolutional layers with an incremental number of filters and kernel sizes of \(3\times 3\), followed by the max-pool \(2\times 2\) operator. In all cases, Batch Normalization [50] and Rectified Linear Unit activations [51] are considered. At the output of this block, two recurrent bidirectional layers of 256 neurons, implemented as LSTM units [52], try to convert the resulting filtered image into a discrete sequence of musical symbols that takes into account both the input sequence and the modeling of the musical representation. Note that each frame performs a classification, modeled with a fully-connected layer with as many neurons as the size of the alphabet plus 1 (the _blank_ symbol necessary for the CTC function). The activation of this neurons is given by a _softmax_ function, which allows interpreting the output as a posterior probability over the alphabet of music symbols [53]. The learning process is carried out by means of stochastic gradient descent (SGD) [54], which modifies the CNN parameters through back-propagation to minimize the CTC loss function. In this regard, the mini-batch size is established to 16 samples per iteration. The learning rate of the SGD is updated adaptively following Adadelta algorithm [55]. Once the network is trained, it is able to provide a prediction in each frame of the input image. These predictions must be post-processed to emit the actual sequence of predicted musical symbols. Thanks to training mechanism with the CTC loss function, the final decoding can be performed greedily: when the symbol predicted by the network in a frame is the same as the previous one, it is assumed that they represent frames of the same and only only one symbol is concatenated to the final sequence. There are two ways to indicate a new symbol is predicted: either the predicted symbol is different from the previous frame or the predicted symbol of a frame is the _blank_ symbol, which indicates that no symbol is actually found. Thus, given an input image, a discrete musical symbol sequence is obtained. Note that the only limitation is that the output cannot contain more musical symbols than the number of frames of the the input image, which in our case is highly unlikely to happen. \begin{table}
\begin{tabular}{l} \hline \hline
**Input (\(128\times W\times 1\))** \\ \hline
**Convolutional Block** \\ \hline Conv\((32,3\times 3)\), MaxPooling\((2\times 2)\) \\ Conv\((64,3\times 3)\), MaxPooling\((2\times 2)\) \\ Conv\((128,3\times 3)\), MaxPooling\((2\times 2)\) \\ Conv\((256,3\times 3)\), MaxPooling\((2\times 2)\) \\ \hline
**Recurrent block** \\ \hline BLSTM\((256)\) \\ BLSTM\((256)\) \\ Dense\((|\Sigma|+1)\) \\ Softmax\(()\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Instantiation of the CRNN used in this work, consisting of 4 convolutional layers and 2 recurrent layers. Notation: Input\((h\times w\times c)\) means an input image of height \(h\), width \(w\) and \(c\) channels; Conv\((n,h\times w)\) denotes a convolution operator of \(n\) filters and kernel size of \(h\times w\); MaxPooling\((h\times w)\) represents a down-sampling operation of the dominating value within a window of size \((h\times w)\); BLSTM(n) means a bi-directional Long Short-Term Memory unit of \(n\) neurons; Dense(n) denotes a dense layer of \(n\) neurons; and Softmax() represents the _softmax_ activation function.