>dists[dists > threshold] = np.inf
matched_keypoints2 = keypoints2[np.argmin(dists, axis=1)]

Even if all the elements in a row are `np.inf` , the `np.argmin` returns the index 0.