Repository: scikit-learn/scikit-learn
Pull request: FIX Incorrect warning when clustering boolean data
Fixes #18996

Input X was being converted to float at the beginning of fit() method. However when a pairwise boolean metric was requested, X was again converted to boolean with multiple warning.

This caused were two issues:
1. The data, if it was already of boolean type, was first converted to float and later to boolean
2. The low level utility function printed the warning multiple times

As a FIX, a check has been put in place in the fit() method itself to let boolean data pass as such. However, when the data needs to be converted to boolean, it is done at this level only with the same warning that was printed earlier, but only once.

Problem statement:
Incorrect warning when clustering boolean data
#### Describe the bug
When clustering data with a metric that requires boolean data, the console fills up with a huge number of `DataConversionWarning` messages telling me the input data needs to be boolean, even though it already is.

#### Steps/Code to Reproduce
```python
import numpy as np
from sklearn.cluster import OPTICS
x = np.random.randint(2, size=(10,5), dtype=np.bool)
labels = OPTICS(metric='rogerstanimoto').fit_predict(x)
```

`x` has dtype `bool` so this ought to be fine, but it prints many repetitions of the message

> /Users/peastman/miniconda3/envs/tf2/lib/python3.7/site-packages/sklearn/metrics/pairwise.py:1765: DataConversionWarning: Data was converted to boolean for metric rogerstanimoto
>  warnings.warn(msg, DataConversionWarning)

When clustering larger datasets, this message can be repeated hundreds of thousands of times.

#### Versions
```
System:
    python: 3.7.6 | packaged by conda-forge | (default, Mar 23 2020, 22:45:16)  [Clang 9.0.1 ]
executable: /Users/peastman/miniconda3/envs/tf2/bin/python
   machine: Darwin-17.7.0-x86_64-i386-64bit

Python dependencies:
          pip: 20.2.4
   setuptools: 49.6.0.post20201009
      sklearn: 0.23.2
        numpy: 1.19.1
        scipy: 1.5.2
       Cython: 0.29.21
       pandas: 1.0.3
   matplotlib: 3.3.2
       joblib: 0.14.1
threadpoolctl: 2.1.0

Built with OpenMP: True
```

Hints from issue discussion:
Thanks for the report.

We do convert `X` to floats at the very beginning of `fit`():
```py
        X = self._validate_data(X, dtype=float)
```

Maybe we should leave it to `bool` if the metric is in `pairwise.PAIRWISE_BOOLEAN_FUNCTIONS`.

Would you like to submit a PR @peastman ?
take

Commit message: FIX- Incorrect warning when clustering boolean data scikit-learn#18996

Patch to review:
diff --git a/sklearn/cluster/_optics.py b/sklearn/cluster/_optics.py
index 8998963704562..b6785693ebee7 100755
--- a/sklearn/cluster/_optics.py
+++ b/sklearn/cluster/_optics.py
@@ -14,6 +14,8 @@
 import warnings
 import numpy as np
 
+from ..exceptions import DataConversionWarning
+from ..metrics.pairwise import PAIRWISE_BOOLEAN_FUNCTIONS
 from ..utils import gen_batches, get_chunk_n_rows
 from ..utils.validation import _deprecate_positional_args
 from ..neighbors import NearestNeighbors
@@ -243,7 +245,13 @@ def fit(self, X, y=None):
         self : instance of OPTICS
             The instance.
         """
-        X = self._validate_data(X, dtype=float)
+
+        dtype = bool if self.metric in PAIRWISE_BOOLEAN_FUNCTIONS else float
+        if dtype == bool and X.dtype != bool:
+            msg = "Data was converted to boolean for metric %s" % self.metric
+            warnings.warn(msg, DataConversionWarning)
+
+        X = self._validate_data(X, dtype=dtype)
 
         if self.cluster_method not in ['dbscan', 'xi']:
             raise ValueError("cluster_method should be one of"


Code review comments:
diff_hunk: @@ -243,7 +245,13 @@ def fit(self, X, y=None):
         self : instance of OPTICS
             The instance.
         """
-        X = self._validate_data(X, dtype=float)
+
+        dtype = bool if self.metric in PAIRWISE_BOOLEAN_FUNCTIONS else float
+        if dtype == bool and X.dtype != bool:
+            msg = "Data was converted to boolean for metric %s" % self.metric
path: sklearn/cluster/_optics.py
text: @user1:
```suggestion
            msg = "Data will be converted to boolean for metric %s" % self.metric
```

@user1:
Also we may add "to avoid this warning, you may convert the data prior to calling fit."

@user1:
Also, please use f-strings instead of the `%` syntax

@author:
thanks for the feedback @user1, dont know how I missed adding the regression tests, but they are there now.