# Fisher Discriminative Sparse Representation Based on DBN for Fault Diagnosis of Complex System
## 3 Fisher Discriminative Sparse Representation (FDSR) of the Features Extracted from DBN
### Optimization of the Fisher Discriminative Sparse Representation  
The FDSR target function is not jointly convex to \((D,X)\); it is convex to each \(D\) and \(X\), whereas the other is fixed. This promotes the alternative optimization algorithm, a sparse coding problem that optimizes \(X\) while \(D\) is fixed. Optimizing \(D\) while \(X\) is fixed is a quadratic programming problem.  
Figure 4: The Fisher discriminative fidelity constraint of a dictionary: \(D_{i}X_{i}\) in (**a**) satisfies the structured dictionary \(D\) well representing \(Y_{i}\). \(D_{i}\) in (**b**) is the optimal subdictionary in \(D_{j}\), \(1\leq j\leq c\) in representing \(A_{i}\). \(X_{i}^{j}\) should have nearly zero coefficients in (**c**) so that representation \(\left\|D_{i}X_{i}^{j}\right\|_{F}^{2}\), \(1\leq j\leq c\) and \(j\
eq i\) are small enough.  
First, we initialized \(D\), applying principle component analysis (PCA) to the feature extracting results of DBN. PCA maps data into a new dimensional subspace that contains that most varied data to ensure the least information loss. An m-dimensional data set \(Y=\left[y_{1},\cdots,y_{n}\right]^{T}\in R^{n\times m}\) includes \(n\) samples, which is given as \(Y=\sum\limits_{i=1}^{m}t_{i}p_{i}^{T}=T_{m}P_{m}^{T}\), where \(t_{k}\)\((k=1,\cdots,m)\) are the score vectors or principal components (PCs), \(p_{k}\)\((k=1,\cdots,m)\) are loading vectors, \(T_{m}\) is the score matrix, and \(P_{m}\) is the loading matrix. Here, we used PCA only to initiate the dictionary but not to reduce the dimensions, so we chose training data from class \(j\)\((j=1,\cdots,c)\) to obtain PCs \(t_{i}^{j}(i=1,\cdots,m)\), and the matrix included \(t_{i}^{j}(i=1,\cdots,m)\) in the corresponding subdictionary \(D_{i}\), in which the initialized dictionary \(D_{i}\) included 100% of the principal components.  
Then, we assumed \(D\) was fixed. The objective function in Equation (14) was reduced as a sparse coding problem to calculate \(X=\left[X_{1},X_{2},\cdots,X_{N}\right]\), as shown in Equation (15), which can be solved using a match pursuit algorithm. In this paper, we solved coefficient \(X_{i}\) class by class. After updating coefficient \(X\), we fixed \(X\) by reducing Equations (13) to (16) to update subdictionary \(D_{i}\) class by class:  
\[J_{X_{i}}=\underset{X_{i}}{\text{argmin}}\left(\begin{array}{c}r(A_{i},D,X_{ i})+\lambda_{1}\left\|X_{i}\right\|_{1}+\\ \lambda_{2}\left(\left\|X_{i}-M_{i}\right\|_{F}^{2}-\sum\limits_{j=1}^{c} \left\|M_{j}-M\right\|_{F}^{2}+\eta\left\|X_{i}\right\|_{F}^{2}\right)\\ \end{array}\right) \tag{15}\]  
\[J_{D_{i}}=\underset{D_{i}}{\text{argmin}}\left(\left\|A-D_{i}X^{i}-\sum \limits_{j=1,j\
eq i}^{c}D_{j}Xj\right\|_{F}^{2}+\left\|A_{i}-D_{i}X_{i}^{j} \right\|_{F}^{2}+\sum\limits_{j=1,j\
eq i}^{c}\left\|D_{j}X_{j}^{i}\right\|_{F }^{2}\right) \tag{16}\]  
where \(X^{i}\) is the coding coefficient of \(A\) over \(D_{i}\).  
To ensure the convexity and discrimination of \(J_{X_{i}}\), we set \(\eta=1\). Then, all terms except \(\lambda_{1}\left\|X_{i}\right\|_{1}\) are differentiable. Equation (15) can be solved using the iterative projection method proposed by [PERSON] et al. [46] (Appendix A). Notably, each column of dictionary \(D_{i}\) should be a unit vector. Equation (16) is a convex optimization problem that can be solved using the algorithm proposed in [PERSON] et al. [47], (Appendix B). The FDSR algorithm is shown in Table 1.