Comparing the Feature Extraction Algorithms for Images | by Sam Bell | Towards Data Science

There are many algorithms out there dedicated to feature extraction of images. Many of them work similarly to a spirograph, or a Roomba. The little bot goes around the room bumping into walls until it, hopefully, covers every speck off the entire floor. Similarly, an algorithm will travel around an image picking up interesting bits and pieces of information from that image. This process is called feature detection.
A good example of feature detection can be seen with the ORB (Oriented FAST and Rotated BRIEF) algorithm. It is actually a hot combination of FAST and BRIEF.
Those markers indicate the important characteristics of that image. This algorithm can even match those features of the same image that has been distorted( grayed, rotated, and shrunk).
ORB essentially finds the “corners” of the image. The FAST component identifies features as areas of the image with a sharp contrast of brightness. If more than 8 surrounding pixels are brighter or darker than a given pixel, that spot is flagged as a feature. Now that we have detected our features, we must express them. BRIEF does this by converting the extracted points as binary feature vectors. These are strings of 128–526 0s and 1s. ORB is pretty useful. I ran into trouble though when it came to applying ORB to a full database of images, and then storing those features into a CSV that would then be used to compare to a given query image in order to find the most similar image. This method is great for any CBIR, but I had difficulty with proper implementation. If any of you have any pointers, please feel free to comment below! :)
Documentation: https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_orb/py_orb.html
Beware! The code at the bottom of the page isn’t actually great. Be sure to use:
orb = cv2.ORB_create()
It may take some clever debugging for it to work correctly.
This method simply measures the proportions of red, green, and blue values of an image and finds an image with similar color proportions. Color gradient histograms can be tuned primarily through binning the values.
Adrian Rosebrock has a great tutorial of implementing this method of comparing images: https://www.pyimagesearch.com/2014/01/22/clever-girl-a-guide-to-utilizing-color-histograms-for-computer-vision-and-image-search-engines/
This method is fine, but it isn’t very detailed. If you query and image with blue skies, it can return ocean images, or images of a pool. Think of it like the color feature in Google Image Search.
A VP-Tree is great if you’re trying to find duplicate images in a database. This algorithm is able to find identical images to the query image, or near-identical images. Again, Adrian Rosebrock has a great tutorial on this: https://www.pyimagesearch.com/2019/08/26/building-an-image-hashing-search-engine-with-vp-trees-and-opencv/
This method essentially analyzes the contents of an image and compresses all that information in a 32-bit integer. Theoretically, the images that have similar compositions would be ordered similarly, and would be neighbors based on composition. This is called hashing, and below is an example.
As you can see, the two images of the sunflower have the same number up to 8 digits. That white text is responsible for the difference, but they would most likely be neighbors. So, if both images were in your dataset one query would result in the other. This algorithm is great for returning identical, or near-identical images. It does not account for the objects in the images being rotated or blurred.
This algorithm is interesting because it seems as though it isn’t an acronym. KAZE refers to the Japanese word for ‘wind.’ Wind flows through “nonlinear forces,” and so, this algorithm is composed of nonlinear diffusion processes in the image domain.
https://www.doc.ic.ac.uk/~ajd/Publications/alcantarilla_etal_eccv2012.pdf
This parallel is a bit of a stretch in my opinion. See these following videos to get a feel for the features KAZE uses.
The last video demonstrates how robust the KAZE model is. KAZE is a great model for identifying the same object in different images. If you had a database of images, like bottles of wine, this would be a good model for label detection, and finding matches based on the label of the wine.
This has been a quick overview of the many different forms of feature extraction for images. There are lots of options available, and each has a different strength to offer for different purposes.
If you are trying to find duplicate images, use VP-trees. KAZE and ORB are great at detecting similar objects in different images. Color histograms are ideal for making one of those pictures made up of thousands of pictures, or at least finding pictures with similar color composition.
Create your own content-based image retrieval system using some of these algorithms, or use a different algorithm! There are so many to choose from. I would love to hear what you come up with.