## 3 Learned Separation
### Complex Objectives  
So far, we have only used the parameters \(c_{j,h}\) from (17) in order to determine the phase values \(\varphi_{j,h}\). However, we can also use them for a different purpose: While the dictionary representation (10) is necessary in order to distinguish the instruments, it is never fully accurate since the relation of the amplitudes of the harmonics can vary slightly even for the same instrument. The typical remedy for this is _spectral masking_, but as explained in [20], this process does not properly deal with interference. Instead, we create a _direct prediction_, in which we replace the tone amplitudes and dictionary entries in (11) with the parameters \(c_{j,h,k}\) (with time-frame dependency added back in):  
\[z_{j}^{\text{dirr}}[k,l] =\sum_{h}c_{j,h,k}\exp\left(-\frac{(\beta l-f_{j,h,k})^{2}}{2 \sigma_{j,k}^{2}}\right), \tag{18a}\] \[z^{\text{dirr}}[k,l] =\sum_{j}z_{j}^{\text{dir}}[k,l]. \tag{18b}\]  
For a fixed \(k\), we set \(y^{\text{dirr}}=z^{\text{dirr}}[k,\cdot]\). The distance function from (12) only considers the absolute value and ignores the phase entirely. This is not necessarily a problem, but knowing the exact phase angles would be useful for resynthesis, and it makes more sense as a training objective. Therefore, we introduce a modified distance function that respects the phase:  
\[d_{2,\delta}^{\text{q,rad}}(Y,y)=\frac{1}{2}\sum_{l}\left|\left([Y[l]]+\delta \right)^{q}\cdot\frac{Y[l]}{[Y[l]]}-\left([y[l]]+\delta\right)^{q}\cdot\frac{ y[l]}{[y[l]]}\right|^{2}. \tag{19}\]  
To avoid division by zero, we add a tiny positive constant to the denominators. We can now compare \(y\) to \(Y\) (as before), \(y^{\text{dir}}\) to \(Y\), and \(y\) to \(y^{\text{dirr}}\), and we can choose between the distance functions \(d_{2,\delta}^{\text{q,abs}}\) and \(d_{2,\delta}^{\text{rad}}\) as defined in (12) and (19). While it would be ideal to have the phases matching in all spectra, there are some caveats:  
* It takes a number of training iterations for \(y_{j}\) to give a useful value. In the meantime, the training of the other parameters can go in a bad direction. * If the discrepancy between \(y\) and \(y^{\text{dir}}\) is high _and_ there is a lot of overlap between the peaks (typically from different tones), the optimal phase values for \(y\) and \(y^{\text{dirr}}\) may be significantly different. An example for this is displayed in Figure 3: The two peaks (red and blue) each have different phases, but by design, those are identical between the predictions. However, since the dictionary prediction is less flexible, its amplitude magnitudes of the harmonics often do not accurately match the input spectrum \(Y\), which shifts the phase in the overlapping region. Thus, attempting to minimize both \(d_{2,\delta}^{\text{q,rad}}(Y,y^{\text{dir}})\) and \(d_{2,\delta}^{\text{q,rad}}(Y,y)\) would lead to a conflict regarding the choice of common phase values. Therefore, we continue to compare \(y\) and \(Y\) via \(d_{2,\delta}^{\text{q,abs}}\) (without the phase), and we use \(d_{2,\delta}^{\text{q,rad}}\) for the comparison between \(y^{\text{dirr}}\) and \(Y\), since \(y^{\text{dirr}}\) is the spectrum that we will end up using for resynthesis, so we aim for the phase to be correct. Between \(y\) and \(y^{\text{dirr}}\), we can compare tone-wise, but since some discrepancy is to be expected, we only associate it with a small penalty; the purpose is to regularize \(y^{\text{dirr}}\) for the case that the peaks of different tones overlap so much that the \(c_{j,h}\) are not unique (see Figure 4). For this task, we employ the loss terms \(d_{2,\delta}^{\text{q,rad}}(y_{j}^{\text{dirr}},y_{j})\) to compare the tone spectra \(y_{j}^{\text{dirr}}:=z_{j}^{\text{dirr}}[k,\cdot]\) and \(y_{j}:=z_{j}[k,\cdot]\) in both magnitude and phase.