Clustering Using OPTICS
By: Colin Sinclair
Tags: Clustering, Data Science, Machine Learning

Clustering Using OPTICS

See What I Did There?

Clustering is a powerful unsupervised knowledge discovery tool used today, which aims to segment your data points into groups of similar features. However, each algorithm is pretty sensitive to the parameters. Similarity based techniques (K-means, etc) are tasked with designating how many clusters exist, while hierarchical usually require manual intervention to decide when to assign finished clusters. The most common density based approach, DBSCAN, requires only two parameters pertaining to how it defines its “Core Points”, but finding the parameters can often be an extremely difficult task. It also will not be able to find clusters of differing densities.

There is a relative of DBSCAN, called OPTICS (Ordering Points to Identify Cluster Structure), that invokes a different process. It will create a reachability plot that is then used to extract clusters and although there is still an input, maximum epsilon, it is mostly introduced only if you would like to try and speed up computation time. The other parameters don’t have as big an effect as their counterparts in other clustering algorithms, and are much easier to use defaults. First, I will explain a little how this algorithm works, how it can include in-line outlier detection, and then how it was very useful for me in a recent application.

The Algorithm

To understand how it creates this plot you must understand a few definitions. You must first understand how DBSCAN works, the parameters it uses, and the difference between core and boundary points. I’ll leave that out of the scope of this article. We’ll also add a couple more definitions

Core Distance- the minimum epsilon to make a distinct point a core point, given a finite MinPts parameter.

Reachability Distance- the reachability-distance of an object p with respect to another object o is the smallest distance from o if o is a core object. It also cannot be smaller than the core distance of o.

Although the MinPts parameter is used in these calculations, the idea is that it would not have much of an impact because all distances would scale at the roughly the same rate.

We will use these definitions to create our reachability plot, which will then be used to extract the clusters. First, we start out by calculating the core distances on all data points in the set. Then we will loop through the entire data set, and update the reachability distances, processing each point only once. We will only update reachability distances of points that stand to be improved, and have not yet been processed. This is because when we process a point, we set in stone its ordering as well as its reachability distance. The next data point chosen to process will be that which has the closest reachability distance. This is how the algorithm keeps clusters near each other in the output ordering. An example of the raw reachability plot is shown below.

The next step will be to extract the actual cluster labels from the plot. The most common way of doing this is by searching for “valleys” in the plot, using local minimums and maximum. A few more parameters could come into play here, depending on the method taken.

See below for a comparison of some generated sample data and the resulting optics labels and reachability plot. Colored points are those identified as clusters, while grey ones represent noise.

This example was taken directly from the Scikit-Learn development version

Notice that there are a good amount of points identified as noise points in this generated example. They have similar densities to that of the yellow cluster, but are not recognized in this extraction because it focuses on separating the denser regions instead. This is where fine tuning the extraction parameters would benefit. Other methods of extracting clusters could also be used in this case.

Outlier Detection

Another interesting aspect of the OPTICS algorithm is an extension of it used for outlier detection, called OPTICS-OF (OF for Outlier Factor). This will give an outlier score to each point, that is a comparison to its closest neighbors rather than the entire set. This is a unique type of outlier detection because of this “local” principle. We can also use some of the previous calculations while creating it.

First, what it does is create a new measure ,“local reachability density” , that is the inverse of the average reachability( with respect to the point you are calculating) of the MinPts-neighbors.

Once we have done this for every point, we will calculate the Outlier Factor. To do so, you take the average of the ratios of the MinPts-neighbors to that specific point, shown below.

The “local” component of OPTICS-OF is key to what separates it apart from other outlier detection methods, as it tries to account for the neighborhood of the specific option. It is also able to give it a relative outlier score, as opposed to just a binary value. An example from the project discussed below is shown, with the outlier objects are shown in red, taking a cutoff of the outlier factor at 2.

One downfall of OPTICS-OF detection is that is cannot handle duplicates well. If enough points occupy the same space, they can have a reach-ability distance of 0, which causes problems in our local outlier factor calculations. In this scenario, I’ve simply dropped all duplicates,

Use Case

I recently used OPTICS for a project that might do a good job of showing where it can be effective, while also giving a shameless plug. I wanted to build an automated knowledge discovery tool in the warehousing/logistics space. A key part of optimizing warehouses comes down to understanding patterns of your SKU base, and how groups of them act similarly while others are completely different. By generalizing an operation too much, you miss opportunity of standardizing and consolidating processes, so clustering is helpful. Forcing every SKU into a group would degrade the accuracy and usefulness of the patterns you are looking to find, so I wanted to take a density based approach. Using DBSCAN would be useful, but will require tuning on each data set, and some patterns would have varying densities and would be harder to find. OPTICS can be used in its case to accompany both of those.

Irregular acting items can also end up causing a significant more amount of labor in comparison to their volume. These are usually slower items you don’t typically pay attention too, and may have inconsistent ordering patterns with other similar SKU’s. These could come from differences in units of handling, seasonality, order commonality, or other factors. But they won’t necessarily be easy to spot because its not one single value that pops out, its the odd combination that does. The Local Outlier Factor was used to try and find these.

Conclusion

Although not a new clustering algorithm by any means, OPTICS is a very interesting technique that I haven’t seen a significant amount of discussion around. It’s advantages include finding varying densities, as well as very little parameter tuning.

References

Ankerst, Mihael, Markus M. Breunig, Hans-Peter Kriegel, and Jörg Sander. “OPTICS: ordering points to identify the clustering structure.” In ACM Sigmod Record, vol. 28, no. 2, pp. 49–60. ACM, 1999.

Ankerst, Mihael, Markus M. Breunig, Hans-Peter Kriegel, and Jörg Sander. “OPTICS-OF: Identifying Local Outliers” in J.M. Zytkow and J. Rauch (Eds.): PKDD’99, LNAI 1704, pp. 262–270, 1999