## 3 Learned Separation
### Training  
The network weights are Glorot-initialized and the biases are initially set to 0. The initial values for the instruments in the dictionary are exponentially decaying sequences along the harmonics:  
\[D_{0}[h,\eta]=\Big{(}\frac{0.5}{\eta}\Big{)}^{h-1}. \tag{29}\]  
We partition the spectra \(Z[k,\cdot]\), \(k=1,\ldots,n_{\text{len}}\), into random batches of size 6. We then train on each batch with the AdaMax algorithm [38]. For each epoch, new random batches are assigned. For the dictionary, we also use AdaMax, but with a reduced learning rate of \(10^{-4}\). Moreover, analogously to [20], for the denominator in AdaMax, we consider the maximum over all the harmonics of the particular instrument when training \(D\). The entire procedure is outlined in Algorithm 1. ```
0:\(Z\), \(\theta\), \(D\) Parameters:\(T\in\mathbb{N}\), \(\kappa_{\theta}>0\), \(\kappa_{D}>0\), \(\beta_{1}\in(0,1)\), \(\beta_{2}\in(0,1)\), \(\varepsilon>0\) \(\gamma_{\theta,1}\gets 0\) \(\gamma_{\theta,2}\gets 0\) \(\gamma_{D,1}\gets 0\) \(\gamma_{D,2}\gets 0\) for\(\tau=1,\ldots,T\)do  choose \(Y\) out of \(\{Z[k,\cdot]:k=1,\ldots,n_{\text{len}}\}\) \(\gamma_{\theta,1}\leftarrow\beta_{1}\gamma_{\theta,1}+(1-\beta_{1})\hat{\xi}_ {\theta,Y}\) \(\gamma_{\theta,2}\leftarrow\max(\beta_{2}\gamma_{\theta,2},|\delta_{\theta,Y}|)\) \(\theta\leftarrow\theta-\frac{\kappa_{\theta}}{1-\beta_{1}}\frac{\gamma_{ \theta,1}}{\gamma_{\theta,2}+\varepsilon}\) \(\gamma_{D,1}\leftarrow\beta_{1}\gamma_{D,1}+(1-\beta_{1})\hat{\xi}_{D,Y}\) \(\gamma_{D,2}\leftarrow\max(\beta_{2}\gamma_{D,2},\max_{h}\lVert\delta_{D,Y}[ h,\cdot]|)\) \(D\gets D-\frac{\kappa_{D}}{1-\beta_{1}^{2}}\cdot\frac{\gamma_{D,1}}{ \gamma_{D,2}+\varepsilon}\) Output:\(\theta,D\)
```  
**Algorithm 1** Training scheme for the network and the dictionary, based on AdaMax [38]. Upper bound regularization of \(D\) and batch summation (see Sections 3.6 and 3.7) are not explicitly stated. ```
0:\(Z\), \(\theta\), \(D\) Parameters:\(T\in\mathbb{N}\), \(\kappa_{\theta}>0\), \(\kappa_{D}>0\), \(\beta_{1}\in(0,1)\), \(\beta_{2}\in(0,1)\), \(\varepsilon>0\) \(\gamma_{\theta,1}\gets 0\) \(\gamma_{\theta,2}\gets 0\) \(\gamma_{D,1}\gets 0\) for\(\tau=1,\ldots,T\)do  choose \(Y\) out of \(\{Z[k,\cdot]:k=1,\ldots,n_{\text{len}}\}\) \(\gamma_{\theta,1}\leftarrow\beta_{1}\gamma_{\theta,1}+(1-\beta_{1})\hat{\xi}_ {\theta,Y}\) \(\gamma_{\theta,2}\leftarrow\max(\beta_{2}\gamma_{\theta,2},|\delta_{\theta,Y}|)\) \(\theta\leftarrow\theta-\frac{\kappa_{\theta}}{1-\beta_{1}^{2 is the _synthesis window_. If the support of \(w\) is cut to \([-1/(2\beta),1/(2\beta)]\), then it is also the _Gabor canonical dual window_ of \(w\), which is the minimum-\(L_{2}\)-norm window to invert the transformation from \(X\) to \(Z\) according to (5). ## 4 Experimental Results and Discussion

We compare our algorithm against two other blind source separation algorithms. We selected them for their ability to identify the sound of musical instruments at arbitrary pitch on a continuous frequency axis. 1. The algorithm from a previous publication of some of the authors [20] assumes an identical tone model, but instead of a trained neural network, it uses a hand-crafted sparse pursuit algorithm for identification, and it operates on a specially computed log-frequency spectrogram. While the data model can represent inharmonicity, it is not fully incorporated into the pursuit algorithm. Moreover, information is lost in the creation of the spectrogram. Since the algorithm operates completely in the real domain, it does not consider phase information, which can lead to problems in the presence of beats. The conceptual advantage of the method is that it only requires rather few hyperparameters and their choice is not critical. 2. The algorithm by [PERSON] et al. [18] detects and clusters peaks in a linear-frequency STFT spectrogram via a probabilistic model. Its main advantage over other methods is that it can extract instrumental music out of a mixture with signals that cannot be represented.