# Fuzzy Rough C-Mean Based Unsupervised CNN Clustering for Large-Scale Image Data
## 2 Fuzzy Rough C-Mean Based Unsupervised CNN Clustering
### Joint Clustering and Representation Learning
#### 2.4.2 Cluster Centroid Updating  
Let \(I=\{I_{1},I_{2},\ldots,I_{n_{x}}\}\) be the set of \(n_{x}\) images. Initially, \(k\) random images are selected from input image set \(I\) and extract their features \(H_{j}\) using the pre-trained UCNN \(H_{j}^{(t)}=\left\{h_{1}^{t},h_{2}^{t},\ldots,h_{k}^{t}\right\}\in H_{\text{FC}}\) from FC layer as initial cluster centroid c in the initial iteration (i.e., \(t\) = 0) by UCNN network. The Fuzzy-Rough C-Mean (FRCM) algorithm [39] is performed to update the cluster centroid by objective function of FRCM, which is  
\[J_{m}(u,h)=\sum_{i=1}^{n_{x}}\sum_{j=1}^{k}\left(u_{ij}\right)^{m}\|I_{i}-h_{ j}\|^{2} \tag{5}\]  
The exponent \(m>1\) is utilized to change the weighting impact of membership values. Updated cluster centroid by FRCM is  
\[h_{j}^{t+1}=c_{j}^{t+1}=\frac{\sum_{i=1}^{n_{x}}\left(u_{ij}^{t}\right)^{m}I_ {i}}{\sum_{i=1}^{n_{x}}\left(u_{ij}^{t}\right)^{m}},\ j=1,2,\ldots,k \tag{6}\]  
In iteration \(t\), the \(j\)th-centroid \(c_{j}^{t}\) that is assigned as a new sample \(h_{j}^{(t)}\), where \(h_{j}^{(t)}\) represent the extracted features for FC layer. ```
Input: Unlabeled \(I_{n_{x}}\) image dataset, Number of cluster \(k\), Randomly select \(k\) image \(I_{nk}\) from \(I_{nx}\), Extract image feature from \(I_{nk}\) images as the initial centroid. Initial cluster center \(c\), at \(t\) = 0, Exponent index \(m\), Threshold parameter \(T\), Stop criterion \(\varepsilon\). Output: Updated cluster centroid \(c\), Updated extracted features for FC layer. 1. Let \(t\) = 0, initialization the cluster center \(c_{j}^{(o)}=h_{j}^{(o)}=\left\{h_{1}^{o},h_{2}^{o},\ldots,h_{k}^{o}\right\} \in H_{\text{FC}}^{0}\), \(j=\{1,2,\ldots,\ k\}\). 2. Assign the data object to the approximation by Equation (4). 3. Compute \(u_{ij}^{(t)}\) according to Equation (1). 4. Compute new cluster center \(c_{j}^{t+1}\) according to Equation (6). 5. Check convergence of the algorithm. If the algorithm converged, stop, else \(t\) = \(t\) + 1 go to step 2. 6. Assigned features for FC layer as \(h_{j}^{(t+1)}=c_{j}^{(t+1)}\). 7. End

#### 2.4.3 Representation Learning

By using unsupervised CNN, the features are extracted for FC layers to generate the features for clustering by the output of the extracted local salient features from CConv layer [74]. To learn the parameters \(\theta(w_{ri},\ w_{ij})\) of FC and softmax layers of UCNN, we utilize SGD process [75] as in Figure 3, where \(w_{ri}\) is the set of weights of layer FC and \(w_{ij}\) is the set of weights of softmax. In order to learn the parameters of layers FC and softmax, we first define the objective function. \[L=\frac{1}{2}\sum_{j=1}^{k}{(\hat{y}_{j}-y_{j})}^{2} \tag{7}\]

where \(k\) is denoted as the number of the cluster, \(\hat{y}_{j}\) is the predicted jth cluster label utilizing UCNN and \(y_{j}\) is the predicted \(j\)th cluster label by using FRCM that is used as a pseudo ground-truth to assist the update of the UCNN clustering model. Then we compute the gradient of the objective function w.r.t \(w_{ri}\) for updating the weights of FC. For this, first use the chain rule to calculate gradient w.r.t \(w_{ij}\) as follows. \[\frac{\partial L}{\partial w_{ij}}=\frac{\partial L}{\partial\hat{y}_{j}}. \frac{\partial\hat{y}_{j}}{\partial u_{j}}.\frac{\partial u_{j}}{\partial w_ {ij}} \tag{8}\]

where \(u_{j}\) is the activation function of the \(j\)th ReLU [76]. The partial derivative of L w.r.t \(\hat{y}_{j}\) is

\[\frac{\partial L}{\partial\hat{y}_{j}}=\hat{y}_{j}-y_{j} \tag{9}\]

and the partial derivative of ReLU w.r.t its \(u\) is

\[\frac{\partial y_{j}}{\partial u_{j}}=\max(\hat{y}_{j},\ 0) \tag{10}\]

The partial derivative of \(u_{j}=\sum_{i=1}^{k}w_{ij}h_{i}\) w.r.t \(w_{ij}\) is

\[\frac{\partial u_{j}}{\partial w_{ij}}=h_{i} \tag{11}\]

Consequently, \(w_{ij}\) can be updated in the \(t\)th iteration by

\[w_{ij}^{(t+1)}=w_{ij}^{(t)}-\eta\big{(}\hat{y}_{j}-y_{j}\big{)}\cdot\max(\hat{ y}_{j},\ 0)\cdot h_{i}^{(t)} \tag{12}\]

where \(\eta\) is the learning rate.