Are These Similar Enough?
By: Pedram Ataee
Tags: Machine Learning, Technology, Artificial Intelligence, Innovation, Data Science

— Anything less than being identical subjects to interpretation.

We as a human have a complex system to interpret similarity. The concept of similarity, regardless of how it must be measured, varies across contexts and problems. If two objects are completely similar, there is no room for interpretation. However, anything less than being identical subjects to interpretation. An object can be anything such as a time series, a document, or an image.

A human can do a lot of processing tasks such as rotation or translation according to the problem requirements if needed. However, this can not be performed by an AI solution without a series of programming and training. For example, how much you think the below images are similar if they are. If the similarity means having similar elements, they are identical; otherwise, they are completely different.

Both images contain similar elements. Are they similar though? Source.

The rabbit and duck illusion is another interesting example. An identical shape with a different angle can be interpreted differently. This shows the significance of angle or rotation in the human interpretation from an image. Therefore, you must always be aware of the sensitivity of an algorithm to the angles of the input data. You sometimes need a rotation-invariant algorithm and sometimes a rotation-variant algorithm.

The duck or rabbit illusion

— We compare objects based on different characteristics.

As said above, anything less than being identical subjects to interpretation. Now, the question is “what are the main characteristics for the similarity?”. If we can properly elaborate on the similarity aspects that we want to measure, it becomes much easier to formulate. Since the first section was explained using image data, I explain this section based on text data to show the high-level concepts are applied to any type of data regardless of its type.

In text-processing, we have three main types of similarity measures including lexical (or form-related), syntactical (or structure-related), and semantical (or meaning-related). Using either of these measures does not disregard the significance of other measures. Two sentences can be similar according to their form but very different according to their meaning.

The most simple textual data is a series of letters or a string that may or may not have meaning. The similarity between two strings can be measured by different methodologies such as edit distance-based or token-based. I do not want to explain how you must implement these methodologies. What I want to emphasize is even the simplest form of similarity can be measured differently. The larger the data length, the more complex the similarity concept. Therefore, you must be prepared to encounter a complex concept of similarity with a larger length in data such as document similarity.

In text-processing, we have three main types of similarity measures including lexical (or form-related), syntactical (or structure-related), and semantical (or meaning-related). Using either of these measures does not disregard the significance of other measures.

— The behavior of the similarity metric must be analyzed properly.

The complexity arises when you want to measure similarity using a mathematical equation. There are various methods to calculate similarity or distance between two objects that I briefly explained in the previous section. In this section, I want to describe the significance of analyzing the behavior of the similarity metric regardless of the calculation method.

In general, we can use similarity and distance interchangeably with some consideration. For example, similarity and distance are opposite of each other so if two objects are very similar it means there is a small distance between them. That is why in many cases similarity(x,y)=1 - distance(x,y) or similarity(x,y)= 1 / distance(x,y). Either of these methods imposes different sensitivity to the similarity metric that becomes crucial while you want to use it in a machine learning algorithm.

A small change in each dimension of the data point that is represented by a vector may significantly skew the vector in high-dimensional space in contrary to two-dimensional space where the vector does not relatively skew.

A similarity metric works differently based on the dimension of space where objects are represented at. For example, if the cosine similarity between two objects in a two-dimensional space is 0.9 you may call those points similar. However, you may not be able to drive the same conclusion in a high-dimensional space with that threshold. Plus, if cos(x, y) = cos(x, z) = 0.9 you barely can make any useful conclusion on the relationship of y and z in a high-dimensional space. On the contrary, you can draw useful conclusions in a two-dimensional space based on spatial geometry. These observations show that you must use the cosine similarity differently based on the space dimension.

The Last Word

In machine learning, especially in clustering techniques, you must constantly work with similarity or distance metrics. For example, one of the main steps in the clustering techniques is to determine the cluster that a new point belongs to. In this scenario, you must calculate the similarity between the new data point and, for example, the cluster centroid and compare it with a threshold.

You can simply use Euclidean distance or cosine distance in many applications. However, it is highly suggested to deeply learn the similarity concept if you want to become an expert in the data science field. I can not say this will solely solve all of your problems but it is an important tool in your toolbox.