# Time Series Prediction Based on Adaptive Weight Online Sequential Extreme Learning Machine
## 2 Preliminaries
### Optimization Extreme Learning Machine  
ELM is a high-efficiency SLFN learning algorithm, where the hidden node parameters can be assigned randomly. Assume that there are \(N\) different training data \(\{(\mathbf{x}_{i},\mathbf{t}_{i})\}_{i=1}^{N}\subset\Re^{n}\times\Re^{m}\) for the supervised learning process, where \(\mathbf{x}_{i}=[x_{i1},x_{i2},\ldots,x_{in}]^{T}\in\Re^{n}\) and \(\mathbf{t}_{i}=[t_{i1},t_{i2},\ldots,t_{im}]^{T}\in\Re^{m}\) are the input vector and output vector, respectively, and the mathematical model of SLFNs is described as:  
\[f_{L}(\mathbf{x})=\sum\limits_{i=1}^{L}\mathbf{\beta}_{i}g(\mathbf{\omega}_{i },b_{i},\mathbf{x}),\mathbf{x}\in\Re^{n} \tag{1}\]  
where \(\mathbf{\omega}_{i}\in\Re^{n}\), \(b_{i}\in\Re\) and \(\mathbf{\beta}_{i}=[\beta_{i1},\beta_{i2},\ldots,\beta_{im}]^{T}\in\Re^{m}\) denote the \(i\)th hidden node parameters, \(L\) is the hidden nodes number, and \(g(\mathbf{\omega}_{i},b_{i},\mathbf{x})\) represents the hidden-layer output in accordance with the input \(\mathbf{x}\). If the \(N\) training samples are absolutely approximated by the SLFNs with \(L\) hidden nodes, it indicates the following equation:  
\[\sum\limits_{i=1}^{L}\mathbf{\beta}_{i}g(\mathbf{\omega}_{i},b_{i},\mathbf{x}) =\mathbf{t}_{j},j=1,2,\ldots,N \tag{2}\]  
Equation (2) is able to be described compactly as the following equation:  
\[\mathbf{H}\mathbf{\beta}=\mathbf{T} \tag{3}\]  
where  
\[\mathbf{H}(\mathbf{\omega}_{1},\ldots,\mathbf{\omega}_{L},b_{1},\ldots,b_{L}, \mathbf{x}_{1},\ldots,\mathbf{x}_{N})=\left[ \begin{array}{ccc}g(\mathbf{ \omega}_{1},b_{1},\mathbf{x}_{1})&\cdots&g(\mathbf{\omega}_{L},b_{L},\mathbf{x }_{1})\\ \vdots&\ddots&\vdots\\ g(\mathbf{\omega}_{1},b_{1},\mathbf{x}_{N})&\cdots&g(\mathbf{\omega}_{L},b_{L}, \mathbf{x}_{N})\\ \end{array} \right]_{N\times L} \tag{4}\]  
\[\mathbf{\beta}=\left[ \begin{array}{c}\mathbf{\beta}_{1}^{T}\\ \vdots\\ \mathbf{\beta}_{L}^{T}\\ \end{array} \right]_{L\times m}\text{ and }\mathbf{T}=\left[ \begin{array}{c} \mathbf{t}_{1}^{T}\\ \vdots\\ \mathbf{t}_{N}^{T}\\ \end{array} \right]_{N\times m} \tag{5}\]  
Traditionally, for the purpose of training an SLFN, one needs to find specific \(\mathbf{\omega}_{i}\), \(b_{i}\), \(\mathbf{\beta}_{i}\), \(i=1,\ldots,L\), such that \(\|\mathbf{H}\mathbf{\beta}-\mathbf{T}\|\) takes a minimum value. If \(\mathbf{H}\) is unknown, the gradient-based approaches are usually employed to iteratively adjust \(\mathbf{\omega}_{i},b_{i}\), \(\mathbf{\beta}_{i}\). However, for most applications, the gradient-based method is extremely time-consuming and often stop at the local minimum. According to the theory of [PERSON], the hidden-layer learning parameters \(\mathbf{\omega}_{i}\) and \(b_{i}\) can be assigned randomly, and as such the SLFN is able to approximate any target function universally as soon as the activation function is nonzero, the target function is continuous and the input sets are compact [17]. If \(L\leq N\), the column rank of \(\mathbf{H}\) is full with probability one, and in real-world applications, the condition \(L\leq N\) can be easily satisfied. Considering the norm of the output weight \(\mathbf{\beta}\) to be part of the cost function [22], the optimization ELM model can be represented as:  
\[ \begin{array}{l}\min:L_{elm}=\frac{1}{2}\|\mathbf{\beta}\|^{2}+\frac{C}{2} \sum\limits_{i=1}^{N}\|\epsilon_{i}\|^{2}\\ st:\mathbf{h}(\mathbf{x}_{i})\mathbf{\beta}=\mathbf{t}_{i}-\epsilon_{i},\;\;i=1,2, \ldots,N\end{array}  \tag{6}\]  
where \(\epsilon_{i}\) is the prediction error of the \(i\)th training sample. Because the convex optimization problem does not have an inequality constraint, the [PERSON]'s condition is satisfied and the strong duality holds.