# A New Fault Diagnosis Method for Unbalanced Data Based on 1 DCNN and L2-SVM
## 3 The Proposed Method
### L2-SVM Loss with Dynamically Adjusted Parameters and Modulation Factors Is Constructed  
The problems of CNN in dealing with unbalanced data classification are analyzed in the above section; however, the solution can be developed with a simple idea: replacing the cross-entropy loss with the L2-SVM loss and improving it accordingly. Since SVM is a generalized linear classifier designed for binary classification, a dynamic adjustment parameter can be added into the binary classification process of SVM, which can be used to impose different penalty factors on normal class samples and fault class samples. Furthermore, we add another modulation factor with focusing parameters to distinguish hard-to-classify samples from easy-to-classify samples. Given a training set \(\mathrm{D}=\{(\mathrm{x}_{1},\mathrm{y}_{1}),(\mathrm{x}_{2},\mathrm{y}_{2}), \ldots,(\mathrm{x}_{n},\mathrm{y}_{n})\},\mathrm{y}_{1}\in\{-1,+1\}\), a partition hyperplane with maximum interval is constructed to separate different types of samples. The basic form of SVM can be expressed as  
\[ \begin{array}{ll}\min_{\mathrm{w,b}}\frac{1}{2}\|\mathrm{w}\|^{2}\\ \mathrm{s.t.}&\mathrm{y}_{i}(\mathrm{w}^{T}\mathrm{x}_{i}+\mathrm{b})\geq 1,i=1,2, \ldots,n\end{array}  \tag{6}\]  
where \(\mathrm{x}_{i}\) is the training sample, \(\mathrm{y}_{i}=\pm 1\) is the category label corresponding to \(\mathrm{x}_{i}\), \(\mathrm{w}\) is the normal vector of the hyperplane, and \(\mathrm{b}\) is the distance between the hyperplane and the origin. While maximizing the interval, the samples that do not meet the constraints should be as few as possible. Therefore, the optimization objective can be written as:  
\[\min_{\mathrm{w,b}}\frac{1}{2}\|\mathrm{w}\|^{2}+c\sum_{i=1}^{n}\ell_{\mathrm{ hinge}}\Big{(}\mathrm{y}_{i}\Big{(}\mathrm{w}^{T}\mathrm{x}_{i}+\mathrm{b} \Big{)}-1\Big{)} \tag{7}\]  
where \(\ell_{\mathrm{hinge}}\) is the hinge loss, and its expression is  
\[\ell_{\mathrm{hinge}}(\mathrm{x})=\max(0,1-\mathrm{x}) \tag{8}\]  
Substituting Equation (8) into Equation (7), we get  
\[\min_{\mathrm{w,b}}\frac{1}{2}\|\mathrm{w}\|^{2}+c\sum_{i=1}^{n}\max(0,1- \mathrm{y}_{i}(\mathrm{w}^{T}\mathrm{x}_{i}+\mathrm{b})) \tag{9}\]  
By introducing \"slack variables\"\(\xi_{i}=\max(0,1-\mathrm{y}_{i}(\mathrm{w}^{T}\mathrm{x}_{i}+\mathrm{b}))\), Equation (9) can be rewritten as:  
\[\min_{\mathrm{w,b},\mathcal{L}_{i}}\frac{1}{2}\|\mathrm{w}\|^{2}+c\sum_{i=1}^{ n}\xi_{i} \tag{10}\]  
where \(\xi_{i}\) is the relaxation factor that represents the error degree of misclassified samples; \(\mathrm{c}\) is used to adjust the error weight of the misclassified samples. Equation (10) is known as L1-SVM, which carries the standard hinge loss but is non-differentiable. The improved Equation (11) is called L2-SVM [38], which is differentiable and imposes a larger loss on the data that violates the boundary and minimizes the sum of squares of the hinge loss. Its expression is as follows:  
\[ \begin{array}{ll}\min_{\mathrm{w,b},\mathcal{L}_{i}}\frac{1}{2}\|w\|^{2}+c \sum_{i=1}^{n}\xi_{i}^{2}\\ \mathrm{s.t.}&\mathrm{yi}(\mathrm{w}^{T}\mathrm{x}_{i}+\mathrm{b})\geq 1-\xi_{i} \\ \mathcal{L}_{i}\geq 0,~{}i=1,2,\ldots,n\end{array}  \tag{11}\]In Equation (11), the equilibrium of classification is operated by the penalty factor c between the maximum interval and the minimum error. The choice of c may cause a large impact on the classification results. When the data are unbalanced due to the normal class samples and the faulty ones have the same penalty factor c, the classification hyperplane of the SVM will be biased toward the class with fewer samples, which leads to the low recognition precision of the minority class samples. Thus, Equation (11) is further modified as follows:  
\[ \begin{array}{l}\min_{w,b,\xi_{4}}\frac{1}{2}\|w\|^{2}+c(1-\eta)\sum\limits_{ i=1}^{n_{\ast}}\xi_{i}^{2}+c(1+\eta)\sum\limits_{i=1}^{n_{\ast}}\xi_{i}^{2}\\ \text{s.t.}\quad\text{yi}(w^{T}x_{i}+b)\geq 1-\xi_{i}\\ \xi_{i}\geq 0,\ i=1,2,\ldots,n\end{array}