Mahalanobis Distance and Multivariate Outlier Detection in R | Towards Data Science

Mahalanobis Distance (MD) is an effective distance metric that finds the distance between point and a distribution (see also). It is quite effective on multivariate data. The reason why MD is effective on multivariate data is because it uses covariance between variables in order to find the distance of two points. In other words, Mahalanobis calculates the distance between point “P1” and point “P2” by considering standard deviation (how many standard deviations P1 far from P2). MD also gives reliable results when outliers are considered as multivariate. In order to find outliers by MD, distance between every point and center in n-dimension data are calculated and outliers found by considering these distances.
Mahalanobis and Euclidean distance
Finding distance between two points with MD
Finding outliers with Mahalanobis distance in R
Conclusions
If you are interested in how to detect outliers using Mahalanobis distance in Python you check my other article below.
towardsdatascience.com
Euclidean distance is also commonly used to find distance between two points in 2 or more than 2 dimensional space. But, MD uses a covariance matrix unlike Euclidean. Because of that, MD works well when two or more variables are highly correlated and even if their scales are not the same. But, when two or more variables are not on the same scale, Euclidean distance results might misdirect. Therefore, Z-scores of variables has to be calculated before finding distance between these points. Moreover, Euclidean won’t work good enough if the variables are highly correlated.
Let’s checkout Euclidean and MD formulas,
As you can see from the formulas, MD uses a covariance matrix (which is at the middle C ^(-1) ) unlike Euclidean. In Euclidean formula p and q represent the points whose distance will be calculated. “n” represents the number of variables in multivariate data.
Finding Distance Between Two Points by MD
Suppose that we have 5 rows and 2 columns data. As you can guess, every row in this data represents a point in 2-dimensional space.
      V1    V2    ----- ----- P1   5     7   P2   6     8   P3   5     6 P4   3     2 P5   9     11
Let’s draw a scatter plot of V1 and V2,
The orange point shows the center of these two variables (by mean) and black points represent each row in the data frame. Now, let’s try to find Mahalanobis Distance between P2 and P5;
According to the calculations above M. Distance between P2 and P5 found 4.08.
As mentioned before MD is quite effective to find outliers for multivariate data. Especially, if there are linear relationships between variables, MD can figure out which observations break down the linearity. Unlike the other example, in order to find the outliers we need to find distance between each point and the center. The center point can be represented as the mean value of every variable in multivariate data.
In this example we can use predefined data in R which is called “airquality”. We will take “Temp” and “Ozone” values as our variable. Here is the list of steps that we need to follow;
Finding the center point of “Ozone” and “Temp”.
Calculating the covariance matrix of “Ozone” and “Temp”.
Finding the Mahalanobis Distance of each point to center.
Finding the Cut-Off value from Chi-Square distribution
Selecting the distances which is less than Cut-Off (These are the values which isn’t an outlier).
Here is the codes to calculate center and covariance matrix;
Before calculating the distances let’s plot our data and draw an ellipse by considering center point and covariance matrix. We can find the ellipse coordinates by using the ellipse function that comes in the “car” package. “ellipse” function takes 3 important arguments; center, shape and radius. Center represents the mean values of variables, shape represents the covariance matrix and radius should be the square root of Chi-Square value with 2 degrees of freedom and 0.95 probability. We take probability values 0.95 because outside the 0.95 will be considered as an outlier and degree of freedom is 2, because we have two variables “Ozone” and “Temp”.
After our ellipse coordinates are found, we can create our scatter plot with “ggplot2” package;
Above, code snippet will return below scatter plot;
Blue point on the plot shows the center point. Black points are the observations for Ozone — Wind variables. As you can see, the points 30, 62, 117, 99 are outside the orange ellipse. It means that these points might be the outliers. If we consider that this ellipse has been drawn over covariance, center and radius, we can say we might have found the same points as the outlier for Mahalanobis Distance. In MD, we don’t draw an ellipse but we calculate distance between each point and center. After we find distances, we use Chi-Square value as Cut-Off in order to identify outliers (same as radius of ellipse in above example).
“mahalanobis” function that comes with R in stats package returns distances between each point and given center point. This function also takes 3 arguments “x”, “center” and “cov”. As you can guess, “x” is multivariate data (matrix or data frame), “center” is the vector of center points of variables and “cov” is covariance matrix of the data. This time, while obtaining Chi-Sqaure Cut-Off value we shouldn’t take square root. Because, MD already returns D2 (squared) distances (you can see it from MD formula).
Finally! We have identified the outliers in our multivariate data. Outliers found 30. 62. 99. 117. observations (rows) same as the points outside of the ellipse in scatter plot.
In this post, we covered “Mahalanobis Distance” from theory to practice. Besides calculating distance between two points from formula, we also learned how to use it in order to find outliers in R. Although MD is not used much in machine learning, it is very useful in defining multivariate outliers.
If you have any question please feel free to leave a comment. However, you may be interested my other article on detecting outliers using Mahalonobis distance in Python from scratch.