What is ISOMAP? How to use ISOMAP? When to use ISOMAP?  What is geodesic distance?  Geodesic distance vs Euclidean distance | Towards Data Science

Dimensionality reduction methods visualize the dataset and reduce its size, as well as reveal different features in the dataset. For non-linear datasets, dimensionality reduction can be examined under various sub-titles such as distance preservation (Isomap), topology preservation (Locally Linear Embedding). This article explains the theoretical part of isometric feature mapping, one of the distance preservation non-linear dimensionality reduction methods, and includes the interpretation and usage areas of the dataset obtained after dimensionality reduction. Studies are enriched with python implementation.
Table of Contents1. Isometric Feature Mapping (ISOMAP)-  Geodesic Distance vs Euclidean Distance-  Interpretation of the result of ISOMAP2. ISOMAP Algorithm3. Use Cases4. Python Tutorials5. References
Isomap is a non-linear dimensionality reduction method, which is a different version of metric MDS, and reduces the dimensionality while preserving geodesic distance. The most obvious difference between Metric MDS and ISOMAP: Euclidean distance is preserved in Metric MDS, while geodesic distance is preserved in ISOMAP. So why do we need this? Is it more effective to preserve geodesic distance? Let’s look at Figure 1 to compare Geodesic distance and Euclidean distance as dimensionality reduction techniques.
While the Euclidean distance calculates only the distance by ignoring the shape of the dataset, the geodesic distance is calculated by passing the shortest path on the dataset. In this case, we can roughly say that the difference between the geodesic distance and the Euclidean distance; while the geodesic distance considers the data adjacent to these data, in the Euclidean distance it is only calculated the shortest linear path. Of course, since our goal is to reduce the dimensionality of the dataset with the least loss, more effective results can be obtained with geodesic distance according to the dataset. The following code block shows the basic implementation of isomap on the Swiss roll dataset:
Looking at the ISOMAP and MDS results, it is seen that the Swiss roll can be unrolled by using ISOMAP.
The red data points are shown as ISOMAP results in Figure 2(right), taking into account the neighborhood distance between them. When calculating the Euclidean distance between 2 points, the shortest distance is obtained (linear distance). In the geodesic distance, the shortest path passing over the dataset is obtained. To find this out, a certain value of k is determined and k-nearest is connected to each other by neighbors and the chain continues. In ISOMAP provided in the Sklearn library, it can be set with n_neighbors=5. The default value is 5. The other way to find the shortest path is to draw the circle centered on the initial data and connect each data point by an edge to all of the other data points within the circle.
To summarize the difference between geodesic distance and Euclidean distance, let’s take a look at their transformed form. Figure 3 illustrates the comparison of tumor tissue which is applied Euclidean distance and geodesic distance.
In Figure 4, the same procedures are applied to the damaged brain region and compared.
In the above images, it is seen that geodesic distance can make much more effective discrimination than the Euclidean distance in image datasets. In Figure 4, geodesic distance has been applied on the brain region image, but it is seen that geodesic distance transform can be used very conveniently for google earth, street, and city datasets due to the structure of the image.
The ISOMAP is applied on the synthetic face dataset and the result is shown in Figure 5.
When looking at the graph, it is seen that the images on the right side of the graph look to the right, the ones on the left to the left, the ones on the upper side look up and the ones below look down so faces turn in the light direction. Another approach is when looking at this graph, it can be easily determined where the camera from which the photo was taken is located. For example, on the left side of the graph, the camera is on the right, on the right side the camera is on the left, on the top side the camera is on the bottom, and the bottom side, the camera is on the top. Of course, these approaches are admissible for this dataset, and each dataset is interpreted individually. In a nutshell, is also a very convenient way to interpret the ISOMAP dataset.
As mentioned above, firstly the weighted graph is constructed -find the nearest neighbors of each sample- This is done in two ways:
As mentioned above, firstly the weighted graph is constructed -find the nearest neighbors of each sample- This is done in two ways:
k nearest neighbors
Fixed radius, define a radius value, and groups the samples which are in the radius value.
After applying one of them above, nearest neighbors are joined via weighted edges. These weights (between the neighbors) will be Euclidean distance. A fully connected weighted graph between the samples is created so far.
2. Using the Floyd Warshall algorithm or Dijkstra’s algorithm, the pairwise distance for all sample pairs in the fully connected weighted graph is calculated using geodesic distance.
3. Multidimensional Scaling (MDS) is applied to get a low dimension version.
The following video explains how to find the shortest path using Floyd- Warshall algorithm:
As a limitation of the algorithm, if k-nearest neighbors or fixed radius is too small, the weighted graph may be interrupted instead of fully connected. If the k value or fixed radius is too high, this time the weighted graph may be too dense; it causes to choose the wrong distance in the manifold.
Isomap can be used in the medical field to make ultrasound & echocardiography images better interpretable with noise removal. The manifold learning algorithm is applied to two-dimensional echocardiography images to discover the relationship between the frames of consecutive cycles of the heart motion.[4] In this approach, each image can be depicted by a point on a reconstructed two-dimensional manifold by the Isomap algorithm and similar points can be related to similar images according to the property of periodic heartbeat cycle stand together.[4]
In a study, ISOMAP was used to interpolate the images between the frames of the videos. By applying ISOMAP to each frame in the video, meaningful features are extracted in the low dimension. To insert new images between 2 frames, it is selected images from the original dataset that are mapped in the same area as those 2 frames.[5]
The articles cited in the reference can be read to learn more about the above use cases.
In the code block below, Isomap is applied to the faces in the fetch_lfw_people dataset and the result is shown in Figure 6.
Dataset License: BSD-3-Clause License
As in the synthetic dataset example above, the faces are sorted according to light.
In the code block below, Isomap has been applied to the number 9 in the MNIST dataset and the result is shown in Figure 7.
Dataset License: BSD-3-Clause License
It is seen that the tail of the number 9 on the left side of the graph is tilted to the right and the tail of the number 9 shifts from right to left as you go to the right in the graph. Similarly, while the tail is short at the top, it gets longer as you go from top to bottom.
ibrahimkovan.medium.com
[1] M. Jordan and J. Kleinberg, Nonlinear Dimensionality Reduction. 2007.
[2] “kimvwijnen/geodesic_distance_transform: (Geodesic) Distance Transform following Toivanen et al (1996).” https://github.com/kimvwijnen/geodesic_distance_transform (accessed Oct. 25, 2021).
[3] “A Global Geometric Framework for Nonlinear Dimensionality Reduction | Request PDF.” https://www.researchgate.net/publication/285599593_A_Global_Geometric_Framework_for_Nonlinear_Dimensionality_Reduction (accessed Oct. 25, 2021).
[4] P. Gifani, H. Behnam, A. Shalbaf, and Z. A. Sani, “Noise reduction of echocardiography images using Isomap algorithm,” 2011 1st Middle East Conf. Biomed. Eng. MECBME 2011, pp. 150–153, 2011, doi: 10.1109/MECBME.2011.5752087.
[5] S. ROBASZKIEWICZ and S. EL GHAZZAL, “Interpolating images between video frames using non-linear dimensionality reduction,” no. x, p. 5, [Online]. Available: http://cs229.stanford.edu/proj2012/ElGhazzalRobaszkiewicz-InterpolatingImagesBetweenVideoFramesUsingNonLinearDimensionalityReduction.pdf.