## 3 Twin Support Vector Machine
### The Solution of TWSVM  
Currently, there are three methods to solve the TWSVM: the Sequential Minimal Optimization (SMO) algorithm [12], the Successive over Relaxation (SOR) iterative method [14; 15] and the Least Squares Twin Support Vector Machine (LSTWSVM) algorithm [16]. The LSTWSVM algorithm is the fastest one, as verified by the experimental analysis, so it is utilized as the solution of TWSVM in this paper.  
Based on the fact that the Support Vectors are almost distributed around the classification hyperplanes, LSTWSVM narrows the constraint of the optimization problem near to the classification hyperplane, and modifies the original optimization models Equations (6) and (7) to Equations (8) and (9), respectively.  
\[\begin{array}{ll}\min_{w_{1},b_{1}}&\frac{1}{2}(Aw_{1}+e_{1}b_{1})^{T}(Aw_ {1}+e_{1}b_{1})+\frac{c_{1}}{2}\xi^{T}\xi\\ s.t&-(Bw_{1}+e_{2}b_{1})+\xi=e_{2}\end{array} \tag{8}\]\[\begin{array}{ll}\min_{w_{2},b_{2}}&\frac{1}{2}(Bw_{2}+e_{2}b_{2})^{T}(Bw_{2}+e_{ 2}b_{2})+c_{2}e_{1}^{T}\xi\\ s.t&(Aw_{2}+e_{1}b_{2})+\xi=e_{1},\quad\xi\geq 0\end{array} \tag{9}\]  
Because of the relaxation factor \(\xi\), the area confirmed by the constraints of Equations (8) and (9) is a rectangle.  
Substituting the constraint into the objective function, Equation (8) becomes:  
\[\min_{w_{1},b_{1}}\ \ \frac{1}{2}\|Aw_{1}+e_{1}b_{1}\|^{2}+\frac{c_{1}}{2}\|Bw_{1 }+e_{2}b_{1}+e_{2}\|^{2} \tag{10}\]  
Differentiating Equation (10) for \(w_{1}\) and \(b_{1}\), respectively, and making the derivatives zero, we can obtain:  
\[A^{T}(Aw_{1}+e_{1}b_{1})+c_{1}B^{T}(Bw_{1}+e_{2}b_{1}+e_{2})=0 \tag{11}\]  
\[e_{1}^{T}(Aw_{1}+e_{1}b_{1})+c_{1}e_{2}^{T}(Bw_{1}+e_{2}b_{1}+e_{2})=0 \tag{12}\]  
Combining Equation (11) with Equation (12), and adapting it into matrix form, then we have:  
\[\left[\begin{array}{cc}B^{T}B&B^{T}e_{2}\\ e_{2}B&m_{2}\end{array}\right]\left[\begin{array}{cc}w_{1}\\ b_{1}\end{array}\right]+\frac{1}{c_{1}}\left[\begin{array}{cc}A^{T}A&A^{T}e _{1}\\ e^{T}A&m_{1}\end{array}\right]+\left[\begin{array}{cc}B^{T}e_{2}\\ m_{2}\end{array}\right]=0 \tag{13}\]  
\[\left[\begin{array}{cc}w_{1}\\ b_{1}\end{array}\right]=\left[\begin{array}{cc}B^{T}B+\frac{1}{c_{1}}A^{T}A &B^{T}e_{2}+\frac{1}{c_{1}}A^{T}e_{1}\\ e_{2}^{T}B+\frac{1}{c_{1}}e_{1}^{T}A&m_{2}+\frac{1}{c_{1}}m_{1}\end{array} \right]^{-1}\left[\begin{array}{cc}-B^{T}e_{2}\\ -m_{2}\end{array}\right] \tag{14}\]  
\[\left[\begin{array}{cc}w_{1}\\ b_{1}\end{array}\right]=\left[\begin{array}{cc}\left[\begin{array}{cc}B^{ T}\\ e_{2}^{T}\end{array}\right]\left[\begin{array}{cc}B^{T}&e_{2}^{T}\end{array} \right]+\\ \frac{1}{c_{1}}\left[\begin{array}{cc}A^{T}\\ e_{1}^{T}\end{array}\right]\left[\begin{array}{cc}A^{T}&e_{1}^{T}\end{array} \right]\end{array}\right]^{-1}\left[\begin{array}{cc}-B^{T}e_{2}\\ -m_{2}\end{array}\right] \tag{15}\]  
Defining \(E=\left[\begin{array}{cc}A&e_{1}\end{array}\right]\) and \(F=\left[\begin{array}{cc}B&e_{2}\end{array}\right]\), Equation (15) becomes:  
\[\left[\begin{array}{c}w_{1}\\ b_{1}\end{array}\right]=-(F^{T}F+\frac{1}{c_{1}}E^{T}E)^{-1}F^{T}e_{2} \tag{16}\]  
Similarly:  
\[\left[\begin{array}{c}w_{2}\\ b_{2}\end{array}\right]=(E^{T}E+\frac{1}{c_{2}}F^{T}F)^{-1}E^{T}e_{1} \tag{17}\]