# Congestive Heart Failure Category Classification Using Neural Networks in Short-Term Series
## 2 Models Based on Neural Networks
### Random Vector Functional Link Network  
The RVFL method was proposed in [51], and, since then, the model has been used to solve problems in various scientific fields due to its learning and generalization characteristics. The structure is as follows: with the random assignment of weights between the input and hidden layers, the RVFL model avoids the backpropagation algorithm, with the only network parameters being the output weights (direct links: input layer to output layer and hidden layer to output layer). Mathematically, RVFL with \(L\) hidden neurons minimizes the following least squares problem [52]:  
\[\min_{\mathbf{\beta}\in\mathbb{R}^{(d+L)\times n}}J(\mathbf{\beta}):=\frac{1}{2}\|\mathbf{H \beta}-\mathbf{T}\|^{2}, \tag{2}\]  
where \(\|\cdot\|\) is the Frobenius norm, \(\mathbf{\beta}\) is the output weights matrix, \(\mathbf{H}=[\mathbf{H}_{1}\mathbf{H}_{2}]_{N\times(d+L)}\) represents the concatenation matrix between the input data and the random output of the hidden layer, and \(T\) is the target matrix, expressed explicitly as follows:  
\[\mathbf{H}_{2}=\left[\begin{array}{cccc}f(\mathbf{w}_{1}\cdot\mathbf{x}_{1}+b_{1})&f( \mathbf{w}_{2}\cdot\mathbf{x}_{1}+b_{2})&\cdots&f(\mathbf{w}_{L}\cdot\mathbf{x}_{1}+b_{L})\\ \vdots&\vdots&\ddots&\vdots\\ f(\mathbf{w}_{1}\cdot\mathbf{x}_{N}+b_{1})&f(\mathbf{w}_{2}\cdot\mathbf{x}_{N}+b_{2})&\vdots &f(\mathbf{w}_{L}\cdot\mathbf{x}_{N}+b_{L})\end{array}\right]_{N\times L},\]  
\[\mathbf{H}_{1}=\left[\begin{array}{cccc}x_{11}&x_{12}&\cdots&x_{1d}\\ \vdots&\vdots&\ddots&\vdots\\ x_{N1}&x_{N2}&\cdots&x_{Nd}\end{array}\right]_{N\times d},\ \mathbf{\beta}=\left[\begin{array}{c}\mathbf{\beta}_{1}\\ \vdots\\ \mathbf{\beta}_{d+L}\end{array}\right]_{(d+L)\times m},\ \text{and}\ \mathbf{T}=\left[\begin{array}{c}\mathbf{t}_{1}\\ \vdots\\ \mathbf{t}_{N}\end{array}\right]_{N\times m}. \tag{3}\]  
Here, the output weights vector \(\mathbf{\beta}_{k}=[\beta_{k1},\beta_{k2},\ldots,\beta_{kL}]^{T}\) connects the \(k\)-th neuron of the input and hidden layers to the output neurons, where \(1\leq k\leq d+L\), and the random weights vector \(\mathbf{w}_{j}=[w_{j1},w_{j2},\ldots,w_{jd}]^{T}\) links the input neurons with the \(j\)-th hidden neuron, \(1\leq j\leq L\). Also, \(f(\cdot)\) is a non-linear activation function, \(b_{j}\) is the bias of the \(j\)-th hidden neuron, and \(x_{i}\cdot x_{j}\) denotes the standard inner product over the space \(\mathbb{R}^{d}\).  
The optimal solution of the standard RVFL model defined in Equation (2) is as follows:  
\[\mathbf{\beta}=\mathbf{H}^{\intercal}\mathbf{T}, \tag{4}\]  
where \(\mathbf{H}^{\intercal}=(\mathbf{H}^{T}\mathbf{H})^{-1}\) is the Moore-Penrose inverse of \(\mathbf{H}\)[53]. In order to design a more stable RVFL model against the overfitting problem, the \(\ell_{2}\)-norm regularizer is added to the function \(J(\cdot)\), resulting in the following convex formulation [54]:  
\[\min_{\mathbf{\beta}\in\mathbb{R}^{(d+L)\times n}}\tilde{J}(\mathbf{\beta}):=\frac{1}{ 2}\|\mathbf{H\beta}-\mathbf{T}\|^{2}+\frac{1}{2}C\|\mathbf{\beta}\|^{2}, \tag{5}\]  
where \(C>0\) is a constant that must be adjusted. Applying efficient optimization strategies, the solution to problem (5) is given as follows:  
\[\begin{split}\text{si}\ (d+L)\leq N:&\mathbf{\beta}= \left(\mathbf{H}^{T}\mathbf{H}+\frac{1}{C}\mathbf{I}\right)^{-1}\mathbf{H}^{T}\mathbf{T}\\ \text{si}\ N<(d+L):&\mathbf{\beta}=\mathbf{H}^{T}\Big{(}\mathbf{H }\mathbf{H}^{T}+\frac{1}{C}\mathbf{I}\Big{)}^{-1}\mathbf{T},\end{split} \tag{6}\]  
where \(\mathbf{I}\) is an identity matrix of proper order. The matrices \(\left(\mathbf{H}^{T}\mathbf{H}+\frac{1}{C}\mathbf{I}\right)\) and \(\left(\mathbf{H}\mathbf{H}^{T}+\frac{1}{C}\mathbf{I}\right)\) in (6) are non-singular matrices, since both \(\mathbf{H}^{T}\mathbf{H}\) and \(\mathbf{H}\mathbf{H}^{T}\) are positive semidefinite symmetric matrices and \(C>0\). When we remove the direct links from inputs to outputs in RVFL, the literature often presents this feedforward neural network as an ELM [55; 56].