Basic Algorithms — Finding the Closest Pair | by Keita Miyaki | Towards Data Science

Writing cost-efficient algorithms is one of the keys to succeed as a data scientist, and in the previous article we used split-conquer method in counting inversions in an array, which is far less costly than brute force method. This time, we will see how another split-conquer algorithm finds the closest pair of points from a set of points on a two-dimensional plane.
Looking up at the sky at Times Square or downtown Tokyo, we would find the closest pair of stars quite easily, because you can see only a few stars there. If you are in the middle of nowhere, the infinite number of stars in the dark night sky would make it impossible to determine the pair.
It is no different for computers; when they determine the closest pairs of points on a plane, the more points the dataset has, the longer it takes for algorithms to find the couple with the least distance. The cost increase is more than linear to the number of points, and we try to write an algorithm which contains the cost growth as low as possible. The split-conquer method works well in this challenge, in a similar way with the algorithm to count inversions.
The brute-force way is, like one that counts inversions in an array, to calculate the distances of every pair of points in the universe. For n number of points, we would need to measure n(n-1)/2 distances and the cost is square to n, or Θ(n2). With two loops, the code for this algorithm can be written as follows.
def find_closest_brute_force(array):        result = {}    result["p1"] = array[0]    result["p2"] = array[1]    result["distance"] = np.sqrt((array[0][0]-array[1][0])**2                                +(array[0][1]-array[1][1])**2)        for i in range(len(array)-1):        for j in range(i+1, len(array)):            distance = np.sqrt((array[i][0]-array[j][0])**2                              +(array[i][1]-array[j][1])**2)            if distance < result["distance"]:                result["p1"] = array[i]                result["p2"] = array[j]                result["distance"] = distance    return resultreturn result
Now we think of a better way whose cost would be O(nlgn). It is possible with presorting and the split-conquer method. To presort the array on one of the coordinates, we also use the slit-conquer method, the merge-sort algorithm.
We sort the arrays with an algorithm called merge-sort, which is faster than brute-force sorting algorithms. The merge-sort algorithm splits the array, sorts the subarrays (as a recursive step), compares the youngest numbers in two subarrays and picks the younger, and repeats it until both subarrays are exhausted. Each of the recursive steps costs just Θ(n) so that the total cost of the algorithm stays at Θ(nlgn).
def merge_sort(array, coordinate=0):        length = len(array)    if length == 1:        return array    if length == 2:        if array[0][coordinate] > array[1][coordinate]:            return np.array([array[1], array[0]])        else:            return array        elif length > 2:        array_l = array[:length//2]        array_r = array[length//2:]        array_l_sorted = merge_sort(array_l, coordinate)        array_r_sorted = merge_sort(array_r, coordinate)                l_length = len(array_l)        r_length = len(array_r)        l = 0        r = 0                sorted_list = []                for i in range(length):            if r == r_length:                sorted_list.append(array_l_sorted[l])                l += 1            elif l == l_length:                sorted_list.append(array_r_sorted[r])                r += 1                                         elif array_l_sorted[l][coordinate] > array_r_sorted[r][coordinate]:                sorted_list.append(array_r_sorted[r])                r += 1                            elif array_l_sorted[l][coordinate] < array_r_sorted[r][coordinate]:                sorted_list.append(array_l_sorted[l])                l += 1                return np.array(sorted_list)
As stated above, we aim to write an algorithm which finds the closest pair of points at a cost of O(nlgn). With a split-conquer algorithm whose recursive steps cost O(n) each would suffice. The algorithm divides the array into subarrays and the key is to see if the closest pair across the two subarrays.
The split conquer algorithm sorts the array by X coordinate, divides the sorted array into two, apply the algorithm recursively to the subarrays, and check whether or not there exists a pair with a shorter distance than that found in subarrays.
def find_closest_nest(array):    X = merge_sort(array, 0)    length = len(X)    if length < 4:        return find_closest_brute_force(array)        else:        array_l = X[:length//2]        array_r = X[length//2:]        dict_l = find_closest_nest(array_l)        dict_r = find_closest_nest(array_r)                            if dict_l["distance"] > dict_r["distance"]:            dict_both = dict_r        else:            dict_both = dict_l                  Y_list = []        for i in range(length):            if X[length//2-1][0]-dict_both["distance"] < array[i][0] < X[length//2-1][0]+dict_both["distance"]:                Y_list.append(array[i])        Y = merge_sort(np.array(Y_list), 1)                      if len(Y) == 1:            dict_final = dict_both        elif len(Y) < 8:            dict_y = find_closest_brute_force(Y)            if dict_both["distance"] > dict_y["distance"]:                dict_final = dict_y            else:                dict_final = dict_both                    else:            for i in range(len(Y)-7):                dict_y = find_closest_brute_force(Y[i:i+7])                                        if dict_both["distance"] > dict_y["distance"]:                    dict_final = dict_y                else:                    dict_final = dict_both            return dict_final
The last step, looking at the pairs across subarrays, needs some tricks to keep the cost at a linear level (e.g. O(n)). First, we will make a subset of the input, which consists of points within d distance from the midpoint in terms of X coordinate; d is the shortest distance between the pair within subarrays. If we find the closest pair across the subarrays, the pair of points should exist within distance d from the line dividing the array into subarrays.
In the example shown in the right hand side, the closest pair within subarrays is determined in the right subarray (note that the point on the dashed line belongs to the left subarray) and its distance is d.
If the closest pair exists across the left and right subarrays, the points should be within the range of d from the dashed line dividing the array into the two subarrays. Therefore, we can look at the subset within the shaded range.
Second, we sort the subset we obtained in the previous step by Y coordinate. We show that we have to look at sets of only eight consecutive points each in the sorted subset.
As shown in the figure, the maximum number of points that can exist in 2d*d rectangle across right and left subarrays is eight (points on the dashed line duplicate; two belong to the left subarray and another two are in the right).
Third, we look at each set of eight consecutive points in the subset sorted on the Y coordinate. If we find a pair whose distance is less than d, it means the closest pair exists across the subarrays. This step costs O(n), and the total cost of this recursive algorithm stays at O(nlgn).
We learned that the split-conquer algorithm we developed performs faster than brute-force one. Let’s compare the actual performance of two algorithms. In the chart below, the cost (execution time) of two algorithms are shown by different sizes of arrays. The two lines clearly indicate that the split-conquer method has an advantage as the sample size increases. This result proves the importance of coding efficiently, as discussed in the previous article too.