How to Search Data with KDTree. Given a Task to Find Data in a Certain... | by Khuyen Tran | Towards Data Science

Imagine you keep a record of the number of travelers in your region every day. You are asked to give a report on the number of travelers from February 2nd to March 2nd. Assuming this data is sorted, how could you find the data within this range in a short amount of time? This is a statement of a range search problem.
This seems like an easy problem. Can we just go through every data from left to right? Yes, but this will take a huge amount of time if we have significant big data, not taking into account that our data could be in k dimensions.
This problem could be generalized to 2 dimensions like a 2D map.
Instead of searching for every centimeter in this space, we could divide this space into a regular grid MxM. Then determine the cells that fall entirely in the range and those that intersect the range.
We will include all the data from the cells that fall entirely within the range and check individual data for all not-fully included cells. The great thing about this method is that we can quickly eliminate the data that is not in the range and narrow down our search.
But what if our data is non-uniform, which means there are clusters of data in some regions and few data in other regions?
If we choose too few grids (M small), we will have many data points in one cell. The performance will get close to one of the sequential search.
If we alternatively choose many grids (M large), we could end up with too many empty cells and waste of space.
Since in real life, we often need to deal with non-uniform data like a map, we need a better approach.
When dealing with clustered data like above, using regular grids will lead to poor behavior. That is why it is ideal to keep this idea of pre-segmenting the data, but segmenting based on the data density!
A small secret that you may not know: The data structure I will introduce is the implementation that creates the robustness to KNearest Neighbor from sklearn instead of the well-know KNearest Neighbor algorithm’s implementation.
Before thinking about k dimensions, let’s start with 2 dimensions. We are given the data points like this
Instead of splitting the data into grids with equal width, we will split the data at the median value at an axis (x or y). We could start by splitting at the median values along the y-axis to divide the plane into half.
Then in each half of the plane, continue dividing the sub-division into two equal sub-division by choosing the vertical lines passing through the median value in each sub-division.
We repeat this recursively the same procedure in the two regions until there is no more point (or the number of points we decide) in the corresponding partition element.
With this method, we would be able to partition the data based on its density! So what is the underlying data structure of this method?
It is very similar to a binary search tree, but not exactly.
Since the data is first split on the y-axis, y is the root node.
The children are x because each half-plane is split on the x-axis.
The left subtree nodes will have all the x-values inferior to the one of the root.
The right subtree nodes will have all their x-values superior to the one of the root.
Alternate between two axes until we reach the limit.
Sounds good so far. But how can we use this data structure to search for data in a particular range?
The good news is that it becomes much simpler to find data once we have created a 2-d tree.
We visit the tree nodes starting from the tree root:
If the node is inside the queried range, add it to the answer
If the queried range intersects the zone defined by the left-subtree, explore it
If the queried range intersects the zone defined by the left-subtree, explore it
Let’s say we have a dataset for locations of ECOBICI’s stations (Mexico’s City bike public sharing system). Our task is to find the data in a certain range. As we can see, there are denser clusters of stations in some regions than others.
To see how the data above could be split using Kdtree, clone this repo
git clone https://github.com/khuyentran1401/kdtree-implementation
Then cd to kdtree-implementation directory in your local machine and run
python ecobi.py
You should see the kd-tree representation of the data above. The regions with more lines indicate the denser clusters in that region.
To find a range in a particular region, run
python findrange.py
You should see the prompt to enter the range. Let’s say if you choose the range between -1 and 1 on the x-axis and -1 to 1 on the y-axis you should see the coordinates of data in that range.
Awesome! With KdTree, you could find the data in a particular range in a matter of seconds. Even though we do not cover how to search for the nearest point with KdTree, KdTree can also be used for Sklearn’s KNearestNeighbor efficiently.
Congratulation! You have learned what KdTree is and its applications. To learn more about how to implement it and use it for your data, check out my Github repo.
I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.
Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these: