## 2 Proposed Framework
### Note Verification  
Note verification for the specific piano is implemented through an NMF. As a frame-based approach, the traditional NMF factorises a spectrogram of a piano signal into 88 spectral bases and sparsity activations. Here the NMF only takes the candidate onsets and pitches into consideration and provides a note-wise representation. In the proposed framework, the sound to be transcribed is reconstructed by:  
\[R_{t-T}^{t+T}=\sum_{k=1}^{K}W_{k}H_{t-T}^{t+T} \tag{3}\]  
where \(R_{t-T}^{t+T}\) is the reconstructed spectrogram of \(2T+1\) frames and \(t\) is the frame of candidate onset. \(W\) is the attack template for the specific piano, \(k\in[1,K]\) is the index of candidate pitches, and \(H_{t-T}^{t+T}\) is the note activations. For the piano to be transcribed, 88 individual notes are pre-recorded and each template is obtained by computing the average spectrum over time frames. The attack template was calculated using the attack stage of each note rather than the whole duration. Note activations  
Figure 3: CNN architecture for onset detection.  
\(H_{t-T}^{t+T}\) can be estimated by minimising the difference between the reconstruction \(R_{t-T}^{t+T}\) and the original spectrogram \(X_{t-T}^{t+T}\). The spectrogram \(X_{t-T}^{t+T}\) is also the input being fed to the pitch estimation CNN. Finally, we verified the candidate notes from activations. Only the candidate pitches whose peaks in the activations exceed a threshold will be identified. Meanwhile, the time when activations exceed the threshold will be set as the onset. Compared with the traditional NMF, the proposed method can save computing time and storage space to a great extent.  
An illustration of note verification is shown in Figure 4. Figure 4a is a spectrogram excerpt used for traditional NMF, in which a C4 note starts at 0.14 s and ends at 0.96 s. Additionally, a C#4 note fades away before the C4 note appears, and a A3 note is played at last. Here, we only present the factorization of note C4. The templates and activations are shown in Figure 4b,c, respectively. Compared with the traditional template (solid line), the attack template (dashed line) concentrates on the percussive stage of the note and shows a different characteristic. For example, both the high-order harmonics and components between harmonics have higher amplitude in the spectrum of the attack template. In Figure 4c, the solid line is the frame-wise activations for traditional NMF, and the dashed line corresponds to the attack activations for note verification. Both curves rise rapidly at the onset time, and a note C4 can be detected using a threshold of 3.0. However, another peak appears in the curve of traditional activations at the end of note C4, and a false positive will be detected using the threshold. Therefore, the NMF using attack templates are more suitable to be applied in note verification.  
In the stage of note verification, the effect of the dynamic level of templates is important. Even for a specific piano, the spectrograms of same pitch vary depending on different dynamics. Figure 5 shows the attack templates of note C4, played at three common dynamics: forte  
Figure 4: An illustration of note verification: (**a**) a spectrogram excerpt used for NMF; (**b**) the attack templates and traditional templates; (**c**) the attack activations and traditional activations in NMF.  
As shown in Figure 5, there are differences between the templates of three dynamics--especially for the higher partials. In the high-frequency range, the notes of louder dynamic have richer spectral content compared to notes of softer dynamics. This indicates that the louder dynamics excite more modes in the vibration of strings than softer dynamics, which is consistent with the assumption of [30]. If we factorize a forte note using piano templates, false positives may happen because the forte note contains some spectral content which is not present in the corresponding piano template. This error will not occur when we transcribe a note using attack templates of louder dynamics.