Repository: scikit-learn/scikit-learn
Pull request: FEA Allow string input for pairwise distances
#### Reference Issues/PRs

closes #15932
closes #17991
closes #24674
supersedes #17991
supersedes #24674

#### What does this implement/fix? Explain your changes.

This allows the user to compute pairwise distance using a custom metric for non-metric data types (e.g. string or boolean).

#### Any other comments?

This PR just implements the changes from #24674 (which has stalled), including the last round of suggestions, and changes the name of the new parameter from `check_length_only` to `only_check_num_samples`.

Problem statement:
compute pairwise_distance with custom metric function for non-numeric data
This is a feature request: I'm working with strings and have a custom metric function to compute similarities between the strings. I would love to use the pairwise_distance function together with this custom metric to compute the whole distance/similarity matrix for my data. However, I'm getting a `ValueError: could not convert string to float: 'some string'` when the X and Y arrays are checked. It would be great if these checks could be made optional for custom metric functions.

Hints from issue discussion:
Thanks for the feature request. Sounds useful indeed (though I have not studied the question in detail). See earlier related discussion in https://github.com/scikit-learn/scikit-learn/issues/3737 and https://github.com/scikit-learn/scikit-learn/issues/4208
Can I work on this request?
You're welcome to work on this @venkyyuvy.

Note that this is only especially helpful for estimators that accept a callable metric, but do not accept a precomputed distance matrix. I'm not sure such estimators exist at the moment in scikit-learn. So this has limited value.
Ya, valid point. Feeding a precomputed distance matrix could a possible solution for this. 

@cod3licious, your opinion about this solution?
I wanted to use the function to compute a distance matrix that I was using elsewhere, i.e., outside of sklearn; I figured the function had some nice parallelization built in or was in some other way more efficient than a naive implementation. So yes, it's probably of limited value in conjunction with sklearn models, but even if there the better solution would be to pass a precomputed distance matrix, this matrix needs to be computed somehow as well. And considering that it's probably just a matter of adding one parameter `check_input=True` and then one if statement before the arrays are checked, I think it's worth it, even if the benefit doesn't extend to other sklearn models.
I'm fine to see this fixed or improved in any case
If nobody is working on this I'd like to take a crack at it
ya, please go ahead. 
@bshabashFD @venkyyuvy are you going to work on this, or can I take over?
I have to add something to this topic. I am the main maintainer of scikit-fda, a project that implements functional data methods compatible with scikit-learn. In our case we do not even have arrays, as our data represent functions, so we have our own objects analog to a 1d array of functions (but sharing common things between them). Also we have developed functional metrics, such as the Lp metrics (which use integrals instead of sums). We even know how to compute the pairwise distance for some of these metrics in a more efficient way than the naive implementation (for example multiplying the weights of the quadrature and using einsum). Moreover, we want to apply some distance-based methods almost verbatim to our objects, such as knn and agglomerative clustering, an objective that we currently achieve wrapping the estimators and using the "precomputed" distance.

In summary, it would be nice if we had support for the following things:

- Computing the pairwise distances with our types and metrics, relying in the optimized implementation if available.
- Using the distance based classifiers directly with our types and metrics, again relying in the optimized implementation of the pairwise distances if available.

Commit message: Update sklearn/metrics/tests/test_pairwise.py

Patch to review:
diff --git a/doc/whats_new/v1.4.rst b/doc/whats_new/v1.4.rst
index 07f4d335058b4..43b654ef511b4 100644
--- a/doc/whats_new/v1.4.rst
+++ b/doc/whats_new/v1.4.rst
@@ -347,6 +347,13 @@ Changelog
 :mod:`sklearn.metrics`
 ......................
 
+- |Feature| :func:`metrics.pairwise_distances` accepts calculating pairwise distances
+  for non-numeric arrays as well. This is supported only with custom functions
+  and the user is expected to pass `only_check_num_samples=True` to be lenient
+  regarding the input validation.
+  :pr:`27456` by :user:`Venkatachalam N <venkyyuvy>`, :user:`Kshitij Mathur <Kshitij68>`
+  and :user:`Julian Libiseller-Egger <julibeg>`.
+
 - |Fix| computing pairwise distances with :func:`euclidean_distances` no longer
   raises an exception when `X` is provided as a `float64` array and
   `X_norm_squared` as a `float32` array. :pr:`27624` by
diff --git a/sklearn/metrics/pairwise.py b/sklearn/metrics/pairwise.py
index 901cccbe46df9..0553da8e710e6 100644
--- a/sklearn/metrics/pairwise.py
+++ b/sklearn/metrics/pairwise.py
@@ -77,6 +77,7 @@ def check_pairwise_arrays(
     dtype=None,
     accept_sparse="csr",
     force_all_finite=True,
+    only_check_num_samples=False,
     copy=False,
 ):
     """Set X and Y appropriately and checks inputs.
@@ -103,10 +104,14 @@ def check_pairwise_arrays(
         Y.
 
     dtype : str, type, list of type, default=None
-        Data type required for X and Y. If None, the dtype will be an
-        appropriate float type selected by _return_float_dtype.
+        Data type required for `X` and `Y`. If `None` and
+        `only_check_num_samples=False`, the `dtype` will be set to an appropriate
+        floating type, preserving the bitness. If `only_check_num_samples=True`, `dtype`
+        will be ignored.
 
         .. versionadded:: 0.18
+        .. versionchanged:: 1.4
+           The parameter is ignored when `only_check_num_samples` is `True`.
 
     accept_sparse : str, bool or list/tuple of str, default='csr'
         String[s] representing allowed sparse matrix formats, such as 'csc',
@@ -130,6 +135,14 @@ def check_pairwise_arrays(
         .. versionchanged:: 0.23
            Accepts `pd.NA` and converts it into `np.nan`.
 
+    only_check_num_samples : bool, default=False
+        Whether to only check the length of the array.
+        When `True`, `dtype` is ignored and the check for 2D array is not performed.
+        This is particularly useful when passing non-numerical inputs and a custom
+        metric.
+
+        .. versionadded:: 1.4
+
     copy : bool, default=False
         Whether a forced copy will be triggered. If copy=False, a copy might
         be triggered by a conversion.
@@ -145,11 +158,13 @@ def check_pairwise_arrays(
         An array equal to Y if Y was not None, guaranteed to be a numpy array.
         If Y was None, safe_Y will be a pointer to X.
     """
-    X, Y, dtype_float = _return_float_dtype(X, Y)
 
+    X, Y, dtype_float = _return_float_dtype(X, Y)
     estimator = "check_pairwise_arrays"
-    if dtype is None:
+    if dtype is None and not only_check_num_samples:
         dtype = dtype_float
+    elif only_check_num_samples:
+        dtype = None
 
     if Y is X or Y is None:
         X = Y = check_array(
@@ -159,6 +174,7 @@ def check_pairwise_arrays(
             copy=copy,
             force_all_finite=force_all_finite,
             estimator=estimator,
+            ensure_2d=not only_check_num_samples,
         )
     else:
         X = check_array(
@@ -168,6 +184,7 @@ def check_pairwise_arrays(
             copy=copy,
             force_all_finite=force_all_finite,
             estimator=estimator,
+            ensure_2d=not only_check_num_samples,
         )
         Y = check_array(
             Y,
@@ -176,6 +193,7 @@ def check_pairwise_arrays(
             copy=copy,
             force_all_finite=force_all_finite,
             estimator=estimator,
+            ensure_2d=not only_check_num_samples,
         )
 
     if precomputed:
@@ -185,12 +203,16 @@ def check_pairwise_arrays(
                 "(n_queries, n_indexed). Got (%d, %d) "
                 "for %d indexed." % (X.shape[0], X.shape[1], Y.shape[0])
             )
-    elif X.shape[1] != Y.shape[1]:
+    elif not only_check_num_samples and X.shape[1] != Y.shape[1]:
         raise ValueError(
             "Incompatible dimension for X and Y matrices: "
             "X.shape[1] == %d while Y.shape[1] == %d" % (X.shape[1], Y.shape[1])
         )
-
+    elif only_check_num_samples and len(X) != len(Y):
+        raise ValueError(
+            f"Incompatible length for X and Y matrices: len(X) == {len(X)} while len(Y)"
+            f" == {len(Y)}"
+        )
     return X, Y
 
 
@@ -1811,9 +1833,16 @@ def _parallel_pairwise(X, Y, func, n_jobs, **kwds):
     return ret
 
 
-def _pairwise_callable(X, Y, metric, force_all_finite=True, **kwds):
+def _pairwise_callable(
+    X, Y, metric, force_all_finite=True, only_check_num_samples=False, **kwds
+):
     """Handle the callable case for pairwise_{distances,kernels}."""
-    X, Y = check_pairwise_arrays(X, Y, force_all_finite=force_all_finite)
+    X, Y = check_pairwise_arrays(
+        X,
+        Y,
+        force_all_finite=force_all_finite,
+        only_check_num_samples=only_check_num_samples,
+    )
 
     if X is Y:
         # Only calculate metric for upper triangle
@@ -2087,17 +2116,29 @@ def pairwise_distances_chunked(
         "metric": [StrOptions(set(_VALID_METRICS) | {"precomputed"}), callable],
         "n_jobs": [Integral, None],
         "force_all_finite": ["boolean", StrOptions({"allow-nan"})],
+        "only_check_num_samples": ["boolean"],
     },
     prefer_skip_nested_validation=True,
 )
 def pairwise_distances(
-    X, Y=None, metric="euclidean", *, n_jobs=None, force_all_finite=True, **kwds
+    X,
+    Y=None,
+    metric="euclidean",
+    *,
+    n_jobs=None,
+    force_all_finite=True,
+    only_check_num_samples=False,
+    **kwds,
 ):
     """Compute the distance matrix from a vector array X and optional Y.
 
     This method takes either a vector array or a distance matrix, and returns
-    a distance matrix. If the input is a vector array, the distances are
-    computed. If the input is a distances matrix, it is returned instead.
+    a distance matrix.
+    If the input is a vector array, the distances are computed.
+    If the input is a distances matrix, it is returned instead.
+    If the input is a collection of non-numeric data (e.g. a list of strings or a
+    boolean array), a custom metric must be passed and `only_check_num_samples` should
+    be set to `True` for distances to be computed.
 
     This method provides a safe way to take a distance matrix as input, while
     preserving compatibility with many other algorithms that take a vector
@@ -2183,6 +2224,14 @@ def pairwise_distances(
         .. versionchanged:: 0.23
            Accepts `pd.NA` and converts it into `np.nan`.
 
+    only_check_num_samples : bool, default=False
+        Whether to only check the length of the array.
+        When `True`, check for 2D array is not performed.
+        This is particularly useful when passing non-numerical inputs and a custom
+        metric.
+
+        .. versionadded:: 1.4
+
     **kwds : optional keyword parameters
         Any further parameters are passed directly to the distance function.
         If using a scipy.spatial.distance metric, the parameters are still
@@ -2220,7 +2269,11 @@ def pairwise_distances(
         func = PAIRWISE_DISTANCE_FUNCTIONS[metric]
     elif callable(metric):
         func = partial(
-            _pairwise_callable, metric=metric, force_all_finite=force_all_finite, **kwds
+            _pairwise_callable,
+            metric=metric,
+            force_all_finite=force_all_finite,
+            only_check_num_samples=only_check_num_samples,
+            **kwds,
         )
     else:
         if issparse(X) or issparse(Y):
diff --git a/sklearn/metrics/tests/test_pairwise.py b/sklearn/metrics/tests/test_pairwise.py
index a9e26fb6ce96e..4c28b2a863c59 100644
--- a/sklearn/metrics/tests/test_pairwise.py
+++ b/sklearn/metrics/tests/test_pairwise.py
@@ -1625,6 +1625,88 @@ def test_numeric_pairwise_distances_datatypes(metric, global_dtype, y_is_x):
     assert_allclose(dist, expected_dist)
 
 
+@pytest.mark.parametrize(
+    "X,Y,expected_distance",
+    [
+        (["a", "ab", "abc"], None, [[0.0, 1.0, 2.0], [1.0, 0.0, 1.0], [2.0, 1.0, 0.0]]),
+        (
+            ["a", "ab", "abc"],
+            [
+                "a",
+                "a",
+                "a",
+            ],
+            [[0.0, 0.0, 0.0], [1.0, 1.0, 1.0], [2.0, 2.0, 2.0]],
+        ),
+    ],
+)
+def test_pairwise_dist_custom_scoring_for_string(X, Y, expected_distance):
+    """
+    Checks the pairwise distance between any two matrices containing strings.
+
+    This is done by passing a metric to calculate the similarity and also setting
+    `only_check_num_samples` to `True` to avoid checks expecting float/int objects in
+    input array.
+    """
+
+    def dummy_string_similarity(x, y):
+        return np.abs(len(x) - len(y))
+
+    actual_distance = pairwise_distances(
+        X=X, Y=Y, metric=dummy_string_similarity, only_check_num_samples=True
+    )
+    assert_array_equal(actual_distance, expected_distance)
+
+
+def test_pairwise_dist_custom_scoring_for_bool():
+    """
+    Checks the pairwise distances between boolean arrays.
+
+    This is done by passing a dummy metric to calculate the distances and also setting
+    `only_check_num_samples` to `True` to avoid the input beig cast to foat.
+    """
+
+    def dummy_bool_dist(v1, v2):
+        # dummy distance func using `&` and thus relying on the input data being boolean
+        return 1 - (v1 & v2).sum() / (v1 | v2).sum()
+
+    X = np.array(
+        [
+            [1, 0, 0, 0],
+            [1, 0, 1, 0],
+            [1, 1, 1, 1],
+        ]
+    ).astype(bool)
+
+    expected_distance = np.array(
+        [
+            [0.0, 0.5, 0.75],
+            [0.5, 0.0, 0.5],
+            [0.75, 0.5, 0.0],
+        ]
+    )
+
+    actual_distance = pairwise_distances(
+        X=X, metric=dummy_bool_dist, only_check_num_samples=True
+    )
+    assert_array_equal(actual_distance, expected_distance)
+
+
+def test_pairwise_distances_raises_error_for_incompatible_length():
+    """Check that we raise an error if `X` and `Y` do not have compatible length."""
+
+    def dummy_string_similarity(x, y):
+        return np.abs(len(x) - len(y))  # pragma: no cover
+
+    X = ["a"]
+    Y = ["a", "a"]
+    msg = "Incompatible length for X and Y matrices"
+    with pytest.raises(ValueError, match=msg):
+        pairwise_distances(
+            X, Y, metric=dummy_string_similarity, only_check_num_samples=True
+        )
+
+
 @pytest.mark.parametrize("csr_container", CSR_CONTAINERS)
 def test_sparse_manhattan_readonly_dataset(csr_container):
     # Non-regression test for: https://github.com/scikit-learn/scikit-learn/issues/7981


Code review comments:
diff_hunk: @@ -130,6 +135,14 @@ def check_pairwise_arrays(
         .. versionchanged:: 0.23
            Accepts `pd.NA` and converts it into `np.nan`.
 
+    only_check_num_samples : bool, default=False
+        Whether to only check the length of the array.
path: sklearn/metrics/pairwise.py
text: @user1:
```suggestion
        Whether to only check for array length consistency.
```

diff_hunk: @@ -77,6 +77,7 @@ def check_pairwise_arrays(
     dtype=None,
     accept_sparse="csr",
     force_all_finite=True,
+    only_check_num_samples=False,
path: sklearn/metrics/pairwise.py
text: @user1:
this API to me is confusing. Why not allow strings and objects as dtype, and have another parameter called `ensure_2d` to skip 2d check explicitly?

@author:
replaced `only_check_num_samples=False` with `ensure_2d=True`

diff_hunk: @@ -185,12 +203,16 @@ def check_pairwise_arrays(
                 "(n_queries, n_indexed). Got (%d, %d) "
                 "for %d indexed." % (X.shape[0], X.shape[1], Y.shape[0])
             )
-    elif X.shape[1] != Y.shape[1]:
+    elif not only_check_num_samples and X.shape[1] != Y.shape[1]:
         raise ValueError(
             "Incompatible dimension for X and Y matrices: "
             "X.shape[1] == %d while Y.shape[1] == %d" % (X.shape[1], Y.shape[1])
         )
-
+    elif only_check_num_samples and len(X) != len(Y):
path: sklearn/metrics/pairwise.py
text: @user1:
why not call `check_consistent_length`?

@author:
The length of the input is checked in any case. So I'd find `check_consistent_length` a little confusing. We could call it `only_check_consistent_length` or `only_check_input_length_consistency` or similar if you prefer.