# Generating Fingerings for Piano Music with Model-Based Reinforcement Learning
## 5 Experiment and Evaluation
### Analysis of Fingering Results  
To facilitate comparison, we conducted experiments using the PIG dataset. Due to differences in input representations, we performed some preprocessing steps. For instance, we treated the space between longer rests as separate sequences and considered arpeggio as chords rather than continuous single notes. We compared our algorithm with the statistical learning algorithm [10] and the DQN reinforcement learning algorithm [5]. Since the DQN implementation only supports monophonic melodies and is based on a simpler environmental model, we removed most of the melodies containing chords and used less data for comparison.  
We evaluated the results using the match rate \(M_{gen}\)[10] for annotation accuracy and the metrics crossed chord, thumbless cross, hop, and step spread [12] for fingering quality. The dataset ground truth is presented in Table 2. The experimental results are presented in Table 3.  
We made an improvement in evaluating the overall hand stretching using the step spread metric. Previously, the method used the semitone difference \(\Delta p\) divided by the difference in finger labels \(\Delta f\). In our approach, we replaced the semitone difference with the difference in white key distances \(\Delta d_{k}\), resulting in \(StepSpread=\Delta d_{k}/\Delta f\). This modification provides a better reflection of the physical distances on the keyboard.  
Our method exhibited a lower match rate \(M_{gen}\) compared to the statistical approach, which was expected considering that reinforcement learning algorithms do not acquire any experience from the PIG dataset. However, our method achieved significant improvement in several metrics that measure poor fingering, such as crossed chord, thumbless cross, and hop. We were able to eliminate nearly all instances of physically impossible fingerings and difficult-to-play fingerings.  
Our method achieved the best results in the evaluation of the overall hand stretching metric, step spread. This indicates that our reinforcement learning algorithm has been optimized for motion efficiency, leading to superior performance in terms of hand stretching.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **Crossed Chord** & **Thumbless Cross** & **Hop** & **Step Spread** \\ \hline Full dataset & 92 & 224 & 561 & 1.18 \\ Test set & 72 & 119 & 323 & 1.18 \\ Test set (De-dupe) 1 & 10 & 18 & 47 & 1.20 \\ \hline \hline \end{tabular} \({}^{\dagger}\) Keeping only one sample for each different piece.  
\end{table}
Table 2: Dataset ground truth.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Method** & **Mgen** & **Crossed Chord** & **Thumbless Cross** & **Hop** & **Step** \\ \hline
1 rd HMM & 61.7 & 25 & 6 & 14 & 1.18 \\
2 rd HMM & 64.3 & 17 & 8 & 44 & 1.17 \\
3 rd HMM & 64.5 & 16 & 9 & 48 & 1.18 \\ DQN & 41.7 & — 1 & 21 & 28 & 1.16 \\ Prioritized sweeping & 58.4 & 0 & 5 & 4 & 1.14 \\ \hline \hline \end{tabular} \({}^{\dagger}\) This implementation cannot label chords.  
\end{table}
Table 3: Experimental result. HMM represents a previous data-driven statistical learning method. DQN is a model-free reinforcement learning algorithm, implemented based on a simpler environmental model. Prioritized sweeping is the model-based reinforcement learning algorithm utilized in this research.