# Efficient Melody Extraction Based on Extreme Learning Machine
## 2 Preliminaries  
### Extreme Learning Machine  
Feedforward neural networks can approximate complex nonlinear mapping functions directly from training data and provide models for difficult artificial phenomena. Traditionally, the parameters of the feedforward networks need to be tuned, leading to the dependency between different layers. The gradient descent-based methods are commonly used for parameter learning. However, these methods are often time-consuming due to either improper steps or converging to local minima. In this subsection, the SLFN and ELM will be presented in detail. Given \(N\) arbitrary distinct samples \((\mathbf{x}_{j},\mathbf{y}_{j})\), where \(\mathbf{x}_{j}=[x_{j1},x_{j2},\ldots,x_{jn}]\in\mathbb{R}^{n}\) and \(\mathbf{y}_{j}=[y_{j1},y_{j2},\ \ldots\,y_{jm}]\in\mathbb{R}^{m}\), standard SLFNs with \(\overline{N}\) hidden nodes and an activation function \(g(\cdot)\) are modeled as:  
\[\mathbf{o}_{j}=\sum_{i=1}^{\overline{N}}\beta_{i\delta i}(\mathbf{x}_{j})= \sum_{i=1}^{\overline{N}}\beta_{i\delta}g(\mathbf{w}_{i}\bullet\mathbf{x}_{j }+b_{i}),j=1,\ldots,N \tag{1}\]  
where \(\mathbf{o}_{j}\) is the output of \(\mathbf{x}_{j}\), \(\mathbf{w}_{i}=[w_{i1},w_{i2},\ldots,w_{jn}]^{T}\) is the input weight vector connecting the \(i\)-th hidden unit and input units, \(\beta_{i}=[\beta_{i1},\beta_{i2},\ldots,\beta_{im}]^{T}\) is the output weight vector connecting the \(i\)-th hidden unit and the output units, \(b_{i}\) is the bias of the \(i\)-th hidden unit, and \(\mathbf{w}_{i}\bullet\mathbf{x}_{j}\) denotes the inner product of \(\mathbf{w}_{i}\) and \(\mathbf{x}_{j}\). The architecture of the ELM network that maps \(\mathbf{x}_{j}\) to \(\mathbf{o}_{j}\) is illustrated in Figure 1.  
\(N\) samples can be approximated by the standard SLFNs with \(\overline{N}\) hidden nodes and an activation function \(g(\cdot)\), i.e.,  
\[\sum_{j=1}^{N}\|\mathbf{o}_{j}-\mathbf{y}_{j}\|=0 \tag{2}\]  
Figure 1: Architecture of the extreme learning machine (ELM) network. In other words, there exist \(\beta_{i}\), \(\mathbf{w}_{i}\) and \(b_{i}\) satisfying  
\[\sum_{i=1}^{\widetilde{N}}\beta_{i}g(\mathbf{w}_{i}\bullet\mathbf{x}_{j}+b_{i}) =\mathbf{y}_{j}\text{, }j=1,\ldots,N \tag{3}\]  
The above \(N\) equations can be denoted in the matrix form as:  
\[\mathbf{H}\beta=\mathbf{Y} \tag{4}\]  
where hidden layer output matrix \(\mathbf{H}\), weight matrix \(\beta\), and label matrix \(\mathbf{Y}\) are, respectively, defined as:  
\[\mathbf{H}= \left[ \begin{array}{ccc}g(\mathbf{w}_{1}\bullet\mathbf{x}_{1}+b_{ 1})&\ldots&g(\mathbf{w}_{\widetilde{N}}\bullet\mathbf{x}_{1}+b_{\widetilde{N} })\\ \vdots&\ldots&\vdots\\ g(\mathbf{w}_{1}\bullet\mathbf{x}_{N}+b_{1})&\ldots&g(\mathbf{w}_{ \widetilde{N}}\bullet\mathbf{x}_{N}+b_{\widetilde{N}})\end{array} \right]_{ \mathbf{N}\times\widetilde{N}} \tag{5}\] \[\beta= \left[ \begin{array}{c}\beta_{1}^{T}\\ \vdots\\ \beta_{N}^{T}\end{array} \right]_{\widetilde{N}\times m}\] (6) \[\mathbf{Y}= \left[ \begin{array}{c}\mathbf{y}_{1}^{T}\\ \vdots\\ \mathbf{y}_{N}^{T}\end{array} \right]_{\mathbf{N}\times m} \tag{7}\]  
If the activation function \(g(\cdot)\) is infinitely differentiable, the number of hidden nodes satisfies \(\widetilde{N}\leq N\). It is proven in [19] that the input weights \(\mathbf{w}_{i}\) and the hidden layer biases \(b_{i}\) can be randomly initialized and then the output matrix \(\mathbf{H}\) can be calculated. Furthermore, the least-squares solution \(\hat{\beta}\) of the linear function denoted in Equation (4) is:  
\[\|\mathbf{H}(\mathbf{w}_{1},\ldots,\mathbf{w}_{\widetilde{N}^{\prime}},b_{1}, \ldots,b_{\widetilde{N}})\hat{\beta}-\mathbf{Y}\|=\underset{\beta}{\text{min}} \|\mathbf{H}(\mathbf{w}_{1},\ldots,\mathbf{w}_{\widetilde{N}^{\prime}},b_{1}, \ldots,b_{\widetilde{N}})\beta-\mathbf{Y}\| \tag{8}\]  
If \(\widetilde{N}=N\), matrix \(\mathbf{H}\) is square and invertible when the input weights \(\mathbf{w}_{i}\) and the hidden layer biases \(b_{i}\) are randomly initialized. In this case, the SLFN can approximate the training samples with zero error.