Remove distance check, we only care if there are enough points