# Polyphonic Piano Transcription with a Note-Based Music Language Model
## 3 Proposed Framework
### Inference Algorithm
\(\text{beam}^{*}\leftarrow\text{new beam object}\) \(\text{beam}.insert(0,\{\},m_{ml})\) for\(n=1\) to \(N\)do \(\text{beam}.\text{mnp}\leftarrow\text{new beam object}\) for\(l,s,m_{ml}\) in beam do for\(k=1\) to \(K\)do \(\text{y}^{\prime}=p_{a}(y_{n}|x_{n})\).\(k\)-th_most_probable() \(\text{y}^{\prime}=\log p_{m}(y^{\prime}|s)+\log p_{g}(y^{\prime}|x_{n})-\log p (y^{\prime})\) \(m_{ml}^{\prime}\leftarrow\text{mm}\) with \(y_{n}:=y^{\prime}\)  beam_tmp_insert(\(l+l^{\prime},\{s,y^{\prime}\},m_{ml}^{\prime}\)) endfor  endfor \(\text{beam\_tmp}\leftarrow\text{min-priority queue of capacity }w^{**}\) \(\text{beam}\leftarrow\text{beam\_tmp}\) endfor return beam.pop() \(*\) Beam object is a queue of triple \(\{l,s,m_{ml}\}\), where at onset \(n\), \(l\) is the accumulated posterior probability \(p(y_{\tau<n}|x_{\tau<n})\), \(s\) is the partial candidate note sequence \(y_{\tau<n}\) and \(m_{ml}\) stands for the music language model taking \(y_{\tau<n}\) as the current input. \(**\) A min-priority queue of fixed capacity \(w\) maintains at most \(w\) highest values. ```  
**Algorithm 1** Global beam search (GBS). ```
0: The acoustic model's outputs \(p_{a}(y_{n}|x_{n})\) at onset \(n\in[1,N]\); The beam width \(w\);  The branching factor \(K\); the threshold \(T\) applied to the acoustic outputs. \(\text{beam}^{*}\leftarrow\text{new beam object}\) \(\text{beam}.insert(0,\{\},m_{ml})\) for\(n=1\) to \(N\)do \(\text{beam}.insert(0,\{\},m_{ml})\) for\(n=1\) to \(N\)do \(\text{beam}.\text{mnp}\leftarrow\text{new beam object}\) \(\text{beam}.\text{mnp}\) for\(l,s,m_{ml}\) in beam do \(\text{beam}.\text{mnp}.insert(l+l^{\prime},\{s,y^{\prime}\},m_{ml}^{\prime})\) endfor else \(l^{\prime}=\log p_{ml}(y^{\prime}|s)+\log p_{g}(y^{\prime}|x_{n})-\log p(y^{ \prime})\) \(m_{ml}^{\prime}\gets m_{ml}\) with \(y_{n}:=y^{\prime}\)  beam_tmp_insert(\(l+l^{\prime},\{s,y^{\prime}\},m_{ml}^{\prime}\)) endfor \(\text{beam\_tmp}\leftarrow\text{min-priority queue of capacity }w^{**}\) \(\text{beam}\leftarrow\text{beam\_tmp}\) endfor return beam.pop() \(*\) Beam object is a queue of triple \(\{l,s,m_{ml}\}\), where at onset \(n\), \(l\) is the accumulated posterior probability \(p(y_{\tau<n}|x_{\tau<n})\), \(s\) is the partial candidate note sequence \(y_{\tau<n}\) and \(m_{ml}\) stands for the music language model taking \(y_{\tau<n}\) as the current input. \(**\) A min-priority queue of fixed capacity \(w\) maintains at most \(w\) highest values. ```  
**Algorithm 2** Local beam search (LBS). ```
0: The acoustic model's outputs \(p_{a}(y_{n}|x_{n})\) at onset \(n\in[1,N]\); The beam width \(w\);  The branching factor \(K\); the threshold \(T\) applied to the acoustic outputs. \(\text{beam}^{*}\leftarrow\text{new beam object}\) \(\text{beam}.insert(0,\{\},m_{ml})\) for\(n=1\) to \(N\)do \(\text{beam}.insert(0,\{\},m_{ml})\) for\(n=1\) to \(N\)do \(\text{beam}.\text{mnp}\leftarrow\text{

## 4 Experiments

### Dataset

The experiments are conducted on the MAPS database [31]. It is a complete piano dataset that contains audio recordings, related aligned MIDI files and annotated text files. There are nine categories of recordings corresponding to different piano types and recording conditions. Each category consists of isolated notes, chords and 30 pieces of music. In the transcription experiments, we only use the full music pieces in MAPS and divide them into training, validation and test splits. To evaluate the performance of the MLM, the training data and test data contain no overlapping contents. Here, we choose the categories \"StbgTGd2\" and \"ENSTDkCl\" as the test set, which consists of 60 musical pieces. Category \"StbgTGd2\" is produced by the default software piano synthesizer, and \"ENSTDkCl\" is obtained from a real Yamaha Disklavier upright piano. In the other seven categories of MAPS, there are 179 pieces of music, which are different from the contents in test data. For these 179 pieces, we select 90% for training (161 pieces) and the remaining 10% for validation (18 pieces). Details for the data partitions are presented in Appendix.