Edit model card

Model description

This is a image clustering model trained after the Semantic Clustering by Adopting Nearest neighbors (SCAN)(Van Gansbeke et al., 2020) algorithm.

The training procedure was done as seen in the example on keras.io by Khalid Salama.

The algorithm consists of two phases:

  1. Self-supervised visual representation learning of images, in which we use the simCLR technique.
  2. Clustering of the learned visual representation vectors to maximize the agreement between the cluster assignments of neighboring vectors.

Intended uses & limitations

The model is intended to show the effective use of self-supervised learning combined with nearest neighbours for (semantic) image clustering.

You can use these clusters to retrieve images of the same class.

Limitations

This model is not supposed to show any superiority to image classification from supervised learning, but as a POC that unsupervised learning is able to cluster similar images together without any labels.

Possible Improvements:

As given by the original author on keras.io, these steps can be taken to improve the accuary further:

  1. increase the number of epochs in the representation learning and the clustering phases;
  2. allow the encoder weights to be tuned during the clustering phase
  3. perform a final fine-tuning step through self-labeling, as described in the original SCAN paper

Training and evaluation data

Training Data

The model was trained using the CIFAR-10 dataset. For training the images were scaled to (32,32,3).

Hyperparameters

For training the following parameters were used:

  • Feature Vector Dimension: 512
  • Projection Units of Head: 128
  • Number of Cluster: 20
  • K-Neighbours: 5

The encoder was not tuned during clustering.

Evaluation

Visualization of highest confidence cluster picks

Visualization of highest confidence cluster picks

Clusters and their respective labels, accuracy and size

Cluster Label Accuracy Size
cluster 0 frog 31.6 % 3582
cluster 1 frog 19.76 % 2348
cluster 2 horse 26.82 % 2983
cluster 3 bird 29.7 % 1532
cluster 4 airplane 39.16 % 3575
cluster 5 ship 22.38 % 2207
cluster 6 automobile 26.41 % 4365
cluster 7 dog 21.09 % 5049
cluster 8 automobile 21.94 % 4093
cluster 9 truck 29.66 % 4639
cluster 10 bird 23.02 % 1455
cluster 11 truck 17.78 % 3937
cluster 12 deer 30.36 % 2635
cluster 13 dog 22.62 % 1950
cluster 14 frog 22.64 % 4391
cluster 15 airplane 26.89 % 2838
cluster 16 ship 34.7 % 2213
cluster 17 ship 17.59 % 1785
cluster 18 cat 16.57 % 1997
cluster 19 deer 27.25 % 2426

Model Plot

View Model Plot

Model Image

Downloads last month
3
Unable to determine this model’s pipeline type. Check the docs .

Space using keras-io/semantic-image-clustering 1