## 3 Improved Firefly Algorithm-Wavelet Support Vector Machine  
### Wavelet Support Vector Machine (WSVM)  
SVM is a supervised machine learning approach which has been used in forecasting, classification and pattern recognition. Depending on the principle of structural risk minimization and solid statistical theoretical foundation, the SVM can achieve satisfactory classification accuracies under a limited number of training samples. When dealing with nonlinear classification, its basic idea is tantamount to map the input vectors into a very high-dimension feature space, and make classification by maximizing the hyper-plane margin. Given a set of training samples \((\mathbf{F}_{i},y_{i})\), \(i=1,\ 2,\ 3,\ \cdots,\ k\), where \(\mathbf{F}_{i}\in R^{d}\) is the input feature vector of an image, \(d\) is the dimension of feature space, \(y_{i}\in R\) is the corresponding output of \(\mathbf{F}_{i}\), indicating which class the training sample belongs to, and \(k\) is the total account of the training samples. In term of nonlinear classification, the hyper-plane is expressed as:  
\[y_{i}\Big{(}w^{T}\psi(\mathbf{F}_{i})+b\Big{)}\geq 1 \tag{16}\]  
where \(w\) is the weight vector; \(\psi(\mathbf{F}_{i})\) is the nonlinear mapping function and \(b\) is the offset coefficient. Based on Equation (16), the margin between different classes can be derived as \(2/\|\mathbf{w}\|\). It is quite evident that maximizing \(2/\|\mathbf{w}\|\) is equivalent to minimizing \(\|\mathbf{w}\|^{2}/2\). Considering the training samples cannot always be perfectly separable, the SVM optimization problem is defined as:  
\[\left\{ \begin{array}{c}\min\limits_{w,b}\frac{1}{2}\|\mathbf{w}\|^{2}+C\sum \limits_{i=1}^{n}\xi_{i}\\ s.t.\ y_{i}\big{(}w^{T}\psi(\mathbf{F}_{i})+b\big{)}\geq 1-\xi_{i}\end{array} \right. \tag{17}\]  
where \(C\) is the penalty factor for the trade-off between maximal margin and tolerable classification errors; \(\xi_{i}\) is the slack variable which allows samples to fall off the margin, and \(\xi_{i}\geq 0\). To solve the convex quadratic programming problem, Lagrange multipliers are introduced in the above equation. As a result, Equation (17) can be rewritten in a dual formulation:  
\[\left\{ \begin{array}{c}\max\limits_{\alpha}\sum\limits_{i=1}^{n}\alpha_{i}- \frac{1}{2}\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}y_{i}y_{j}x_{i}\alpha_{j} K\big{(}\mathbf{F}_{i},\mathbf{F}_{j}\big{)}\\ s.t.\ 0\leq\alpha_{i}\leq C,\ \sum\limits_{i=1}^{n}\alpha_{i}y_{i}=0\end{array} \right. \tag{18}\]  
where \(\alpha_{i}\) is the Lagrange multiplier of every training sample, and \(\alpha_{i}\geq 0\); \(K\big{(}\mathbf{F}_{i},\mathbf{F}_{j}\big{)}\) is the kernel function, and \(K\big{(}\mathbf{F}_{i},\mathbf{F}_{j}\big{)}=\psi(\mathbf{F}_{i})^{T}\psi(\mathbf{F}_{j})\). As a straightforward computation technology, the kernel function is employed to simplify the mapping process. This method enables to calculate the inner product in the feature space which acts as a function to the original input vectors. The results obtained from high-dimension feature space correspond to the outcomes of the original input space. Assume that the solution of the above equation is \(\alpha_{i}^{*}\), the corresponding classification function is represented as:  
\[f(\mathbf{F})=\text{sgn}\left(\sum\limits_{i=1}^{n}\alpha_{i}^{*}y_{i}K(\mathbf{F}_{i}, \mathbf{F})+b^{*}\right) \tag{19}\]  
where \(b^{*}=y_{j}-\sum\limits_{i=1}^{N}\alpha_{i}^{*}y_{i}K(\mathbf{F}_{i},\mathbf{F}_{j})\). The output of function \(f(\mathbf{F})\) is the result of combustion state recognition. From Equations (18) and (19), it is quite clear that the key technology of SVM is the kernel function, which has a direct effect on the generalization ability and learning ability of the model. In theory, any asymmetric kernel function satisfying [PERSON]'s condition can be introduced. Generally, the common kernel functions are linear, polynomial, sigmoid and radial basis functions. Wavelet technology has unique advantages for classification and approximation of non-stationary signals.