Repository: scipy/scipy

Issue:
scipy-v1.0.0 pdist with metric=`minkowski` raises `ValueError: array must not contain infs or NaNs`
After upgrading scipy to v1.0.0, I started running into this error - `ValueError: array must not contain infs or NaNs` for pdist with metric=`minkowski`

### Reproducing code example:
```
import numpy as np
from scipy.spatial import distance as spatial_distance

arr = np.array([[83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 13500.0, 1350.0, 24000.0, 4800.0, 36000.0, 7200.0, 17896.96875753512, 3579.393751507024, 26845.45313630268, 5369.090627260539, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5], [83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 13500.0, 1350.0, 24000.0, 4800.0, 36000.0, 7200.0, 18869.563687446636, 3773.9127374893255, 28304.345531169954, 5660.869106233963, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5], [83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 13500.0, 1350.0, 24000.0, 4800.0, 36000.0, 7200.0, 15243.337419662861, 3048.667483932572, 22865.006129494293, 4573.001225898869, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5], [83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 8100.0, 1350.0, 24000.0, 4800.0, 60000.0, 7200.0, 15298.724594411382, 3059.7449188822766, 22948.086891617073, 4589.617378323415, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5], [83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 13500.0, 1350.0, 24000.0, 4800.0, 36000.0, 7200.0, 8274.075838530227, 4137.037919265114, 13500.0, 6205.5568788976525, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5]])


spatial_distance.pdist(arr, metric='minkowski', p=1, w=None, V=None, VI=None)
```

### Error message:
```
ValueErrorTraceback (most recent call last)
<ipython-input-10-71b743bead3a> in <module>()
----> 1 spatial_distance.pdist(arr, metric = 'minkowski', p=1, w=None, V=None, VI=None)

.../lib/python2.7/site-packages/scipy/spatial/distance.pyc in pdist(X, metric, *args, **kwargs)
   1732         elif mstr.startswith("test_"):
   1733             if mstr in _TEST_METRICS:
-> 1734                 dm = pdist(X, _TEST_METRICS[mstr], **kwargs)
   1735             else:
   1736                 raise ValueError('Unknown "Test" Distance Metric: %s' % mstr[5:])

.../venv/lib/python2.7/site-packages/scipy/spatial/distance.pyc in pdist(X, metric, *args, **kwargs)
   1688         for i in xrange(0, m - 1):
   1689             for j in xrange(i + 1, m):
-> 1690                 dm[k] = metric(X[i], X[j], **kwargs)
   1691                 k = k + 1
   1692 

.../venv/lib/python2.7/site-packages/scipy/spatial/distance.pyc in minkowski(u, v, p, w)
    479             root_w = np.power(w, 1/p)
    480         u_v = root_w * u_v
--> 481     dist = norm(u_v, ord=p)
    482     return dist
    483 

.../venv/lib/python2.7/site-packages/scipy/linalg/misc.pyc in norm(a, ord, axis, keepdims)
    127     """
    128     # Differs from numpy only in non-finite handling and the use of blas.
--> 129     a = np.asarray_chkfinite(a)
    130 
    131     # Only use optimized norms if axis and keepdims are not specified.

.../venv/lib/python2.7/site-packages/numpy/lib/function_base.pyc in asarray_chkfinite(a, dtype, order)
   1213     if a.dtype.char in typecodes['AllFloat'] and not np.isfinite(a).all():
   1214         raise ValueError(
-> 1215             "array must not contain infs or NaNs")
   1216     return a
   1217 

ValueError: array must not contain infs or NaNs
```

### but metric= `euclidean` works:
```
spatial_distance.pdist(arr, metric='euclidean', p=None, w=None, V=None, VI=None)

array([  1788.09402275,   4878.64185582,  25059.48821966,  16483.68376158,
         6666.73587857,  25460.91149939,  18217.06765729,  24600.21075028,
        11837.38716763,  27341.24826582])
```
### Scipy/Numpy/Python version information:
```
$python -c 'import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)'
('1.0.0', '1.13.3', sys.version_info(major=2, minor=7, micro=10, releaselevel='final', serial=0))
```




Discussion:
older version does not give this error:
```
$pip install scipy==0.19.1

 Found existing installation: scipy 1.0.0
    Uninstalling scipy-1.0.0:
      Successfully uninstalled scipy-1.0.0
Successfully installed scipy-0.19.1

```
```
import numpy as np
from scipy.spatial import distance as spatial_distance

arr = np.array([[83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 13500.0, 1350.0, 24000.0, 4800.0, 36000.0, 7200.0, 17896.96875753512, 3579.393751507024, 26845.45313630268, 5369.090627260539, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5], [83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 13500.0, 1350.0, 24000.0, 4800.0, 36000.0, 7200.0, 18869.563687446636, 3773.9127374893255, 28304.345531169954, 5660.869106233963, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5], [83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 13500.0, 1350.0, 24000.0, 4800.0, 36000.0, 7200.0, 15243.337419662861, 3048.667483932572, 22865.006129494293, 4573.001225898869, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5], [83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 8100.0, 1350.0, 24000.0, 4800.0, 60000.0, 7200.0, 15298.724594411382, 3059.7449188822766, 22948.086891617073, 4589.617378323415, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5], [83.33333333333334, 100.0, 83.33333333333334, 100.0, 36.0, 60.0, 90.0, 150.0, 24.0, 48.0, 60.0, 120.0, 10800.0, 1800.0, 27000.0, 2700.0, 5400.0, 900.0, 13500.0, 1350.0, 24000.0, 4800.0, 36000.0, 7200.0, 8274.075838530227, 4137.037919265114, 13500.0, 6205.5568788976525, 1515.0, 505.0, 2600.75, 2146.25, 2082.0, 694.0, 3574.1000000000004, 2949.5]])

spatial_distance.pdist(arr, metric = 'minkowski', p=1, w=None, V=None, VI=None)
```
gives
```

array([  2917.78478973,   7960.89401362,  37194.73248937,  24362.4564747 ,
        10878.67880335,  40112.51727911,  26307.64633453,  29566.16152425,
        19055.19379896,  48565.96814846])
```

Hmm, I can reproduce that and `arr` doesn't contain any nans/infs. @apbard would you have any time to look into this?
weird...I'll look into this asap.

Resolving patch:
diff --git a/scipy/spatial/distance.py b/scipy/spatial/distance.py
--- a/scipy/spatial/distance.py
+++ b/scipy/spatial/distance.py
@@ -273,8 +273,6 @@ def _validate_mahalanobis_kwargs(X, m, n, **kwargs):
 
 
 def _validate_minkowski_kwargs(X, m, n, **kwargs):
-    if 'w' in kwargs:
-        kwargs['w'] = _convert_to_double(kwargs['w'])
     if 'p' not in kwargs:
         kwargs['p'] = 2.
     return kwargs
@@ -327,7 +325,7 @@ def _validate_vector(u, dtype=None):
     return u
 
 
-def _validate_weights(w, dtype=None):
+def _validate_weights(w, dtype=np.double):
     w = _validate_vector(w, dtype=dtype)
     if np.any(w < 0):
         raise ValueError("Input weights should be all non-negative")


Test patch:
diff --git a/scipy/spatial/tests/test_distance.py b/scipy/spatial/tests/test_distance.py
--- a/scipy/spatial/tests/test_distance.py
+++ b/scipy/spatial/tests/test_distance.py
@@ -1812,6 +1812,18 @@ def test_hamming_string_array():
     assert_allclose(whamming(a, b), desired)
 
 
+def test_minkowski_w():
+    # Regression test for gh-8142.
+    arr_in = np.array([[83.33333333, 100., 83.33333333, 100., 36.,
+                        60., 90., 150., 24., 48.],
+                       [83.33333333, 100., 83.33333333, 100., 36.,
+                        60., 90., 150., 24., 48.]])
+    pdist(arr_in, metric='minkowski', p=1, w=None)
+    cdist(arr_in, arr_in, metric='minkowski', p=1, w=None)
+    pdist(arr_in, metric='minkowski', p=1)
+    cdist(arr_in, arr_in, metric='minkowski', p=1)
+
+
 def test_sqeuclidean_dtypes():
     # Assert that sqeuclidean returns the right types of values.
     # Integer types should be converted to floating for stability.