# Deep Subspace Clustering with Block Diagonal Constraint
## 2 Deep Subspace Clustering Model with Block Diagonal Constraint
### Model Formulation
Thus, \(\|Z\|_{\kappa}\) measures the \(k\) block diagonal degree of \(Z\), and we can minimize \(\|Z\|_{\kappa}\) for learning the representation \(Z\) with \(k\) block diagonal structure. While introducing the above \(k\) block diagonal regularization in ([PERSON] and [PERSON], 2019), we can rewrite the DSC-Net loss function as follows:  
\[\underset{\Theta}{\text{min}}\quad\frac{1}{2}\|X-X^{{}^{\prime}}\|_{F}^{2}+ \frac{1}{2}\|F-FZ\|_{F}^{2}+\alpha_{1}\|Z\|_{F}^{2}+\alpha_{2}\|Z\|_{\kappa}. \tag{3}\]  
Figure 1: Architecture of the proposed Deep Subspace Clustering with Block Diagonal Constraint (DSC-BDC). It consists of mainly three modules: (a) the convolutional encoder module is used to extract convolutional features, and the parameter is marked as \(\theta_{e}\); (b) the convolutional decoder module is used with the encoder module to initialize the convolutional module, and the parameter is marked as \(\theta_{d}\); and (c) the self-expression module with indirectly block diagonal constraint is used to learn a better representation matrix from the self-expression coefficients, by applying a separation strategy to the representation matrix. We know that \(\|Z\|_{\kappa}=0\) will cause a strict block diagonal structure, whose elements are zero except for the diagonal block elements. This is a strong constraint and will make the network over-fit and unstable for increasing error. Therefore, this paper introduces a separation strategy to separate the self-expression matrix for providing more freedom to guarantee both the exact self-expression and approximating \(k\) block diagonal structure. Different from directly constraining \(Z\), this separation strategy decomposes \(Z\) into \(C\) and \(D\), and then we only constrain \(C\) with a block diagonal structure to relax the constraints. We define \(Z=C\odot D\) to call for two distinct matrices, \(C\) and \(D\), with the same size, here \(Z_{ij}=D_{ij}\cdot C_{ij}\), which denotes the element-wise product. We bound the magnitudes of \(C\) via the constraint \(C\leq C_{max}\), which is to avoid the magnitude of \(C\) dominating the other variables. For \(D\), it is easy to see that \(D_{ij}=0\) will make \((C\odot D)_{ij}=0\), even if \(C_{ij}\
eq 0\). Therefore, we use \(D_{min}\) as the lower bound to constrain the elements in \(D\). Motivated by the above idea, we improve DSC-Net model to our deep subspace clustering model with block diagonal constraint (DSC-BDC), which can be formulated as follows:  
\[ \begin{split}\min_{\bar{\Theta},C,D}&\frac{1}{2}\| X-X^{{}^{\prime}}\|_{F}^{2}+\frac{1}{2}\|F-FZ\|_{F}^{2}+\frac{\alpha_{1}}{2}\|Z\|_{F} ^{2}\\ &+\frac{\alpha_{2}}{2}\|Z-C\odot D\|_{F}^{2}+\frac{\alpha_{3}}{2 }\|D\|_{F}^{2}+\alpha_{4}\|C\|_{\kappa}\\ \text{s.t.}& C\in\mathcal{C},D\in\mathcal{D},\end{split}  \tag{4}\]  
where  
\[\mathcal{C}=\{C\in\mathbb{R}^{N\times N}:0\leq C\leq C_{max},diag(C)=0,C=C^{T}\}, \tag{5}\]  
\[\mathcal{D}=\{D\in\mathbb{R}^{N\times N}:D_{ij}\geq D_{min}\}. \tag{6}\]  
\(Z\in\mathbb{R}^{N\times N}\) is the representation matrix corresponding to the subspace structure of \(X\); \(C\in\mathbb{R}^{N\times N}\) and \(D\in\mathbb{R}^{N\times N}\) are obtained by matrix multiplication decomposition of representation matrix \(Z\); and \(\alpha_{1},\alpha_{2},\alpha_{3},\alpha_{4}\) are the regular term parameters. The first term of objective function is the reconstruction errors of the auto-encoder network, the second term is the self-expression error and the third term is the regular term to avoid trivial solutions. For the last three items, \(\|Z-C\odot D\|_{F}^{2}\) is the multiplicative decomposition term, \(\|D\|_{F}^{2}\) is the regularization term and the template \(C\) is constrained as a block diagonal structure through the \(\|C\|_{\kappa}\) term. Thus, the representation matrix \(Z\) will be with an approximate block diagonal structure, and it is hoped this improves the clustering performance.