# Polyphonic Piano Transcription with a Note-Based Music Language Model
## 3 Proposed Framework
### Inference Algorithm
We select the threshold that maximizes the F-measure over the validation set and use the threshold of 0.5 for the following testing. In the proposed architecture, the other two methods are implemented using simple RNN MLMs. As mentioned in Section 3, the GBS algorithm searches for the partial solutions at each detected onset, whereas the proposed inference algorithm predicts notes only at the blank onsets in the thresholding transcription results. Experimental results on the software piano \"StbgTGd2\" are presented in Table 2. In Table 2, we display the note-based recall, precision and F-measure for systems using the three post-processing methods. The acoustic model with the simplest thresholding yields a high F-measure over 90%, which indicates that the CNNs are effective in onset detection and multi-pitch estimation. Compared with the thresholding method, the global decoding post-processing of GBS results in worse transcription on the F-measure. The transcriptions produced by the GBS contain fewer notes, so the recall is lower than that of the thresholding results. This is probably due to the MLM, which is trained to predict notes using the true history. In the GBS algorithm, we take the previous \(w\) candidate solutions as the history, which are estimated using outputs of the acoustic model and the MLM. The drawback of prediction accumulates over time, so that the performance of transcription is unsatisfactory. The proposed algorithm yields a better performance than the GBS on the recall and F-measure, since the determined notes at non-blank onsets can help reduce the accumulation of errors. The improvement of recall is at the cost of a loss of precision. The proposed algorithm also outperforms the thresholding method on recall, which illustrates that the note-based MLM could model note sequences to some extent. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Post-Processing** & **Recall** & **Precision** & **F-Measure** \\ \hline Thresholding & 0.8839 & 0.9169 & 0.9001 \\ GBS (RNN) & 0.8592 & 0.9184 & 0.8878 \\ Proposed (RNN) & 0.8946 & 0.9111 & 0.9027 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Transcription results on the software piano “StbgTGd2”. GBS, global beam search. Table 3 displays the transcription results on the real piano \"ENSTDkCl\". As shown in Table 2, a similar trend can be seen in Table 3, where the best performance is achieved using the proposed algorithm. All the note-based metrics in Table 3 are worse than those in Table 2. This is because the notes produced by the real piano are not as regular as the notes from the software. Additionally, there are some deviations and noises when the real piano is played. Therefore, there are many accumulated errors in the decoding of GBS. This partly explains why the GBS generates the worst results on all metrics. Table 4 presents the transcription results on the LabROSA dataset of real piano recordings. In Table 4, the differences between the results of these three post-processing methods are obvious. We can draw the same conclusion that the proposed inference algorithm improves the performance of transcription using the RNN MLM. Figure 3 shows the threshold's influence on the performance of the thresholding method and proposed algorithm. The threshold of 0.5 is reasonable for the three test sets. We also observe that the performance difference between the thresholding method and the proposed algorithm increases with the increase of the threshold. A higher threshold value will bring more blank onsets, so the superiority of the proposed algorithm for the thresholding method is more obvious. Through Tables 2-4 and Figure 3, we also observe that the superiority of the proposed algorithm compared to the other two methods is more obvious when the acoustic model has a poorer performance in transcription. At the threshold of 0.5, we further perform a paired \(t\)-test over 10-fold cross-validation on the MAPS dataset. The \(t\)-test is used to check whether the proposed algorithm outperforms the thresholding on the F-measure.