# Melody Extraction Using Chroma-Level Note Tracking and Pitch Mapping
## 2 Chroma-Level Note Tracking and Pitch Mapping-Based Melody Extraction
### Melodic Pitch Mapping  
The possible pitch frequencies corresponding to each pitch class are  
\[f_{p}=2^{(\frac{p_{j}}{2}+N_{oct})}f_{ref} \tag{11}\]  
where \(N_{oct}\) takes integers that force \(f_{p}\) to be located within the possible melody pitch range, and \(p_{j}\) is the HPCP pitch bin. After all possible frequencies are obtained by Equation (11), coarse note mapping determines which frequency is the accurate pitch according to the following criterion  
\[f_{0,c,t}=\underset{f_{p}}{\mathrm{argmax}}\sum\limits_{t=t_{on}}^{t_{off}} \sum\limits_{h=1}^{N_{h}}\big{|}A_{h,t}\big{|}^{2}\sum\limits_{h=1}^{N_{h_{2}} }\min(A_{h,t},A_{h+1,t}) \tag{12}\]  
where \(A_{h,t}\) is the magnitude of the \(h\)-th harmonic for candidate \(f_{p}\) at frame \(t\), and \(N_{h_{1}}<N_{h_{2}}\cdot\sum\limits_{h=1}^{N_{h_{1}}}\big{|}A_{h,t}\big{|}^{2}\) in Equation (12) accounts for the salience of candidate frequencies, while \(\sum\limits_{h=1}^{N_{h_{2}}}\min(A_{h,t},A_{h+1,t})\) for spectral flatness. The pitches mapped by Equation (11) are standard frequencies in the equal tempered scale. However, most pitches have marginal shifts from these standard frequencies in real-world recordings. Hence, a fine tuning is necessary to obtain more accurate pitch frequencies. Accurate pitch frequency can be obtained by the modified Euclidean algorithm based on the magnitudes and frequencies of the harmonics [12]. The detailed fine tuning procedure is described as follows. Suppose that \(f_{i,t}\) and \(f_{j,t}\) are frequencies of the \(i\)-th and \(j\)-th peaks at frame \(t\), respectively. Let \(y=\max(f_{i,t},f_{j,t})\), and \(x=\min(f_{i,t},f_{j,t})\). Compute  
\[r(x,y)=\Big{|}\frac{y}{x}-[\frac{y}{x}]\Big{|} \tag{13}\]  
where \([\cdot]\) represents rounding towards the nearest integer. If \(r(x,y)\geq\zeta\), \(z=\mathrm{mod}f(y,x)\), where \(\zeta\) is the threshold determining whether a pitch candidate is deduced, \(\mathrm{mod}f(\cdot,\cdot)\) means modulus after division, and both input variables are float numbers. Then, let \(y=\max(z,x)\), \(x=\min(z,x)\), iterate the above procedure again. If \(r(x,y)<\zeta\), the estimated pitch is  
\[f_{0,a,t}=\frac{x+y}{1+[y/x]} \tag{14}\]A salience value is assigned to the estimated pitch, i.e.,  
\[s(f_{0,\alpha,t})=\sum_{h=1}^{N_{h_{1}}}\big{|}A_{h,t}\big{|}^{2}\sum_{h=1}^{N_{ h_{2}}}\min(A_{h,t},A_{h+1,t}) \tag{15}\]  
where \(A_{h,t}\) is the magnitude of the \(h\)-th harmonic for candidate \(f_{0,\alpha,t}\), and \(N_{h_{1}}<N_{h_{2}}\). For detailed information of the above procedure, please refer to Algorithm 1[12]. ```
1: For \(t=1,\ldots,N_{frm}\), \(n_{t}\) is the number of peaks at frame \(t\), do
2: For \(i,j=1,\ldots,n_{t}\) (\(i\
eq j\)), do
3:\(x=f_{i,t},y=f_{j,t}\), where \(f_{i,t}\) and \(f_{j,t}\) are the \(i\)-th and \(j\)-th spectrum peak frequencies at frame \(t\), respectively. \begin{tabular}{l l} (1) & \(y\leftarrow\max(x,y)\), \(x\leftarrow\min(x,y)\) ; \\ (2) & Compute \(r(x,y)=\big{|}\frac{y}{x}-\big{|}\frac{y}{x}\big{|}\); \\ (3) & If \(r(x,y)\geq\zeta\), then \(z=\max\text{ord}(y,x)\), \(y=z\), where \(\text{modf}(y,x)\) returns modulus after division \(\frac{y}{x}\),  then go to step (1); if \(r(x,y)<\zeta\), then pitch candidate is given according to Equation (14), and \(f_{0,\alpha,t}\) is assigned with a corresponding weight according to Equation (15). If there are still peak pairs which haven't been evaluated, select another pair and go back to step (1); \\ (4) & Output pitch candidates and their related weights, i.e. \(\theta=\{F_{t},S_{F_{t}}\}\), where \(F_{t}=\{f_{0,\alpha,t}\}\) and \(S_{F_{t}}=\{s(f_{0,\alpha,t})\}\). \\ \end{tabular} 
4: End for. 5: End for. ```  
**Algorithm 1** Computation for the parameters of pitch candidates: \(\theta_{t}=\{F_{t},S_{F_{t}}\}\). There are numerous pitch candidates calculated according to the above procedure. Fine tuning aims to track the dynamic changing due to frequency mistuning or singing voice.