# Korean Passori Vocal Note Transcription Using Attention-Based Segmentation and Viterbi Decoding
## 3 Method
### Note-Level Transcription  
The pre-trained model trained on the MedleyDB dataset [33] for vocal melody extraction in the source task provides the meaningful features for estimating F0 values in the target Pansori data. The vocal notes in the target task can be achieved after applying the post-processing in the Pansori estimated F0 of the source task in frame-level transcription. To conduct post-processing for the vocal notes in the Pansori performance, the Viterbi decoding technique is employed, a core process within hidden Markov models (HMMs). In the Viterbi decoding, the transition matrix \(T\) and observation matrix \(P\) play a specific role. More specifically, \(T\) represents the probabilities of transitioning between different hidden states and \(P\) represents the likelihood of observations given a particular state. By utilizing the Viterbi algorithm with \(T\) and \(P\), the goal is to find the most likely sequence of hidden states (Pansori vocal notes) that best explains the observed musical events. Given the definition of states, \(S_{0}\) for the silence state, \(S_{i}\) for the onset state for a particular note \(i\), and \(S_{i}^{\prime}\) for the sustain state for note \(i\), the transition matrix \(T\) with dimensions \(N\times N\) is constructed with specific probabilities for transitions between these states. \(P\) is calculated as \(size_{rows}=2(note_{max}-note_{min})+1\). Similarly, to calculate the size of the columns for the matrix \(P\), the length of F0 values is calculated, which is obtained from our pre-trained model of the source task. To convert the Pansori audio data into MIDI format, first, we passed the wave file of Pansori into our best vocal melody pre-trained model. The F0 for the Pansori vocal data can then be estimated using this pre-trained model. We also calculated the tuning frequency and adjusted the estimated F0. This corrected F0 was transformed to MIDI notes after applying the tuning correction. The tuning frequency aims to refine the accuracy of the estimated pitch by adjusting it based on any detected tuning discrepancies in the audio signal. The librosa library was employed for this purpose, contributing to the overall accuracy of the pitch estimation process [34]. To construct the observation matrix \(P\), we defined the probability of \(voice_{acc}\), \(onset_{acc}\), and \(pitch_{acc}\). If the predicted F0 value by the pre-trained model was non-zero, we assumed that the corresponding frame is considered the voiced frame. But, if the generated F0 value was zero, it was considered the non-voiced frame. We also detected the onset from the audio, and if the time frame was detected as an onset, it set the high probability for the onset state. But, if the time frame was detected as the non-onset state, it set the low probability for the non-onset state. To assign the probabilities to the sustain states of the observation matrix \(P\), we looped over all possible MIDI note values from A2 to E6, and if it exactly matched with the transformed MIDI values, we set the high probability for the sustain state at that time frame. Similarly, if this did not match, we set the low probability for the sustain state. The Viterbi algorithm works by considering the transition matrix \(T\) and observation probabilities \(P\) to identify the most probable sequence of the hidden state. For each time step in the observed sequence (along columns of \(P\)), the Viterbi algorithm assesses all possible state sequences at that time step. It computes the likelihood or probability of each state sequence occurring. It iterates through the observed sequence, updating the probability of arriving at each state based on the maximum likelihood computed from the previous time step. At each step, the algorithm computes the most probable path (sequence of states) by considering both the probability of the current observation given the state and the probability of transitioning to the current state from the previous states.