# Deep Image Clustering Based on Label Similarity and Maximizing Mutual Information across Views
## 3 Deep Image Clustering under Similarity and Mutual Information
### Extended Mutual Information  
The discriminative class-level features have been obtained in the above section. In this section, by exploring the correlation between the features from multiple views of the same image, e.g., original and transformed images, the essential information in the image is learned, and the intra-class diversities in the instance-level representation feature space are also reduced. Maximizing MI between the complete input and the encoder output is not sufficient to learn useful representations [40]. The average MI between the global representation and the local features of the input plays a stronger role in improving the quality of representation than the global MI. Inspired by DIM, this paper only uses local MI and considers that KL divergence has no upper bound theoretically, so it is replaced by JS divergence (JSD) similar to f-gan [43]. The JSD version of MI is defined as:  
\[MI(f_{map},z)=\max E_{j}[-sp(-T(f_{map},Z))]-E_{M}[sp(T(f_{map},Z))] \tag{6}\]  
where \(f_{map}\) corresponds to the local features (feature map), \(Z\) corresponds to the global feature (instance-level representation), \(sp\) is the softplus function and \(T\) is a discriminator trained to distinguish whether \(f_{map}\) and \(Z\) are sampled from the joint distribution \(J\) or from the marginal distribution \(M\). In this paper, only \(f_{map}\) and \(Z\) belong to the same cluster and follow \(J\); otherwise, they follow \(M\). A pair of samples that satisfy the joint distribution is called a pair of positive samples; otherwise, a pair of negative samples. In order to force the instance-level feature \(z\) to capture the essential information, our model extends DIM by introducing data augmentation techniques. Let \(x\) and \(\bar{x}\) be the original and transformed samples, respectively, \(\{C_{i,j}(x)|i=1,2,\ldots,h;j=1,2,\ldots,w\}\) and \(\{\tilde{C}_{i,j}(\bar{x})|i=1,2,\ldots,h;j=1,2,\ldots,w\}\) correspond to their feature map. The negative samples of the feature map are all represented by the symbol \(C^{{}^{\prime}}_{i,j}\). By concatenating \(C_{i,j}\) and \(z\) to get a larger feature map as the input of the discriminator, the estimation of the global-local MI loss can be obtained by minimizing the following loss function:  
\[ \begin{split}& L_{LMI}(\bar{z},C_{i,j})\\ &=-\max I(\bar{z},C_{i,j})=\min-(E_{j}[-sp(-T(\bar{z},C_{i,j}))]- E_{M}[sp(T(\bar{z},C^{{}^{\prime}}_{i,j}))])\\ & L_{LMIT}(\bar{z},\tilde{C}_{i,j})\\ &=-\max I(\bar{z},\tilde{C}_{i,j})=\min-(E_{j}[-sp(-T(\bar{z}, \tilde{C}_{i,j}))]-E_{M}[sp(T(\bar{z},C^{{}^{\prime}}_{i,j}))])\end{split}  \tag{7}\]  
Meanwhile, not only the global-local MI is calculated. The original images and the augmented images' intermediate layer features should also be related, and their MI should be as large as possible. The estimation of local-local MI loss can be obtained by minimizing the following loss function:  
\[ \begin{split}& L_{LMIC}(\tilde{C}_{i,j},C_{i,j})\\ &=-\max I(\tilde{C}_{i,j},C_{i,j})=\min-(E_{j}[-sp(-T(\tilde{C}_{ i,j},C_{i,j}))]-E_{M}[sp(T(\tilde{C}_{i,j},C^{{}^{\prime}}_{i,j}))])\end{split}  \tag{9}\]  
By maximizing the MI between features extracted from multiple views with the same information, it implies that the features need to capture high-level information with decisive influence, such as the existence of an object, so the extension of this paper is reasonable. Since DIM selects negative samples randomly, it is unreasonable to minimize the MI of pairwise samples when they belong to the same cluster. Therefore, in a similar way to DCCM, the positive and negative pairs of the same sample are selected according to the similarity of the sample pairs in this batch. Similar pairwise samples are regarded as positive pairs; otherwise, they are negative pairs. Maximizing MI between positive pairs promotes intra-class aggregation. Minimizing MI between negative pairs promotes inter-class separation.