Repository: pydata/sparse

Issue:
Cryptic error message during a large broadcast.
```py
>>> np.array([[1, 2], [3, 4]])*sparse.COO.from_numpy([1, 1])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/aaronmeurer/anaconda3/lib/python3.6/site-packages/sparse/coo/core.py", line 1441, in __array_ufunc__
    result = elemwise(ufunc, *inputs, **kwargs)
  File "/Users/aaronmeurer/anaconda3/lib/python3.6/site-packages/sparse/coo/umath.py", line 47, in elemwise
    return _Elemwise(func, *args, **kwargs).get_result()
  File "/Users/aaronmeurer/anaconda3/lib/python3.6/site-packages/sparse/coo/umath.py", line 423, in __init__
    self._check_broadcast()
  File "/Users/aaronmeurer/anaconda3/lib/python3.6/site-packages/sparse/coo/umath.py", line 511, in _check_broadcast
    .format(non_ndarray_shape))
ValueError: All ndarrays must be broadcastable to the shape without ndarrays (2,)
>>> np.array([[1, 2], [3, 4]])*np.array([1, 1])
array([[1, 2],
       [3, 4]])
```

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/pydata/sparse/pull/232?src=pr&el=h1) Report
> Merging [#232](https://codecov.io/gh/pydata/sparse/pull/232?src=pr&el=desc) into [master](https://codecov.io/gh/pydata/sparse/commit/bc81375eb36e845b33b2d1b5cfd6252a1805c75d?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pydata/sparse/pull/232/graphs/tree.svg?width=650&token=H212u0Uxxw&height=150&src=pr)](https://codecov.io/gh/pydata/sparse/pull/232?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master     #232   +/-   ##
=======================================
  Coverage   97.61%   97.61%           
=======================================
  Files          11       11           
  Lines        1549     1549           
=======================================
  Hits         1512     1512           
  Misses         37       37
```


| [Impacted Files](https://codecov.io/gh/pydata/sparse/pull/232?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sparse/coo/umath.py](https://codecov.io/gh/pydata/sparse/pull/232/diff?src=pr&el=tree#diff-c3BhcnNlL2Nvby91bWF0aC5weQ==) | `97.13% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pydata/sparse/pull/232?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pydata/sparse/pull/232?src=pr&el=footer). Last update [bc81375...487cec0](https://codecov.io/gh/pydata/sparse/pull/232?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/.travis.yml b/.travis.yml
index 5935617..4abbf03 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,22 +1,21 @@
-sudo: False
-
 language: python
 
 cache: pip
 
 matrix:
   include:
-    - python: 2.7
     - python: 3.5
     - python: 3.5
       env: NUMPY_VERSION="<1.14.0"
     - python: 3.6
+    - python: 3.7
+      dist: xenial
 
 install:
   - ./ci/01-install.sh
 
 script:
-  - py.test
+  - pytest
 
 after_success:
   - codecov
diff --git a/benchmarks/benchmark_coo.py b/benchmarks/benchmark_coo.py
index 58c050e..39088fb 100644
--- a/benchmarks/benchmark_coo.py
+++ b/benchmarks/benchmark_coo.py
@@ -3,7 +3,7 @@ import numpy as np
 import sparse
 
 
-class ElemwiseSuite(object):
+class ElemwiseSuite:
     def setup(self):
         np.random.seed(0)
         self.x = sparse.random((100, 100, 100), density=0.01)
@@ -21,7 +21,7 @@ class ElemwiseSuite(object):
         self.x[5]
 
 
-class ElemwiseBroadcastingSuite(object):
+class ElemwiseBroadcastingSuite:
     def setup(self):
         np.random.seed(0)
         self.x = sparse.random((100, 1, 100), density=0.01)
@@ -34,7 +34,7 @@ class ElemwiseBroadcastingSuite(object):
         self.x * self.y
 
 
-class IndexingSuite(object):
+class IndexingSuite:
     def setup(self):
         np.random.seed(0)
         self.x = sparse.random((100, 100, 100), density=0.01)
diff --git a/ci/environment-2.7.yml b/ci/environment-3.7.yml
similarity index 78%
rename from ci/environment-2.7.yml
rename to ci/environment-3.7.yml
index 8635938..45826ec 100644
--- a/ci/environment-2.7.yml
+++ b/ci/environment-3.7.yml
@@ -1,8 +1,8 @@
-name: py27-sparse-test
+name: py37-sparse-test
 channels:
   - conda-forge
 dependencies:
-  - python=2.7
+  - python=3.7
   - pytest
   - numpy
   - scipy
diff --git a/docs/contributing.rst b/docs/contributing.rst
index e94e56a..82fde31 100644
--- a/docs/contributing.rst
+++ b/docs/contributing.rst
@@ -60,17 +60,14 @@ Running/Adding Unit Tests
 It is best if all new functionality and/or bug fixes have unit tests added
 with each use-case.
 
-Since we support both Python 2.7 and Python 3.5 and newer, it is recommended
-to test with at least these two versions before committing your code or opening
-a pull request. We use `pytest <https://docs.pytest.org/en/latest/>`_ as our unit
-testing framework, with the ``pytest-cov`` extension to check code coverage and
-``pytest-flake8`` to check code style. You don't need to configure these extensions
-yourself. Once you've configured your environment, you can just ``cd`` to
-the root of your repository and run
+We use `pytest <https://docs.pytest.org/en/latest/>`_ as our unit testing framework,
+with the ``pytest-cov`` extension to check code coverage and ``pytest-flake8`` to
+check code style. You don't need to configure these extensions yourself. Once you've
+configured your environment, you can just ``cd`` to the root of your repository and run
 
 .. code-block:: bash
 
-   py.test
+   pytest
 
 This automatically checks code style and functionality, and prints code coverage,
 even though it doesn't fail on low coverage.
@@ -80,7 +77,7 @@ Unit tests are automatically run on Travis CI for pull requests.
 Coverage
 --------
 
-The ``py.test`` script automatically reports coverage, both on the terminal for
+The ``pytest`` script automatically reports coverage, both on the terminal for
 missing line numbers, and in annotated HTML form in ``htmlcov/index.html``.
 
 Coverage is automatically checked on CodeCov for pull requests.
diff --git a/readthedocs.yml b/readthedocs.yml
index 111706a..0413384 100644
--- a/readthedocs.yml
+++ b/readthedocs.yml
@@ -2,7 +2,7 @@ build:
   image: latest
 
 python:
-  version: 3.6
+  version: 3.7
   pip_install: true
   extra_requirements:
     - docs
diff --git a/setup.py b/setup.py
index 08bc614..33c3897 100755
--- a/setup.py
+++ b/setup.py
@@ -56,11 +56,11 @@ setup(
         'Operating System :: OS Independent',
         'License :: OSI Approved :: BSD License',
         'Programming Language :: Python',
-        'Programming Language :: Python :: 2',
-        'Programming Language :: Python :: 2.7',
         'Programming Language :: Python :: 3',
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3 :: Only',
         'Intended Audience :: Developers',
         'Intended Audience :: Science/Research',
     ],
@@ -69,5 +69,5 @@ setup(
         'Source': 'https://github.com/pydata/sparse/',
         'Tracker': 'https://github.com/pydata/sparse/issues',
     },
-    python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4',
+    python_requires='>=3.5, <4',
 )
diff --git a/sparse/compatibility.py b/sparse/compatibility.py
deleted file mode 100644
index 671c79c..0000000
--- a/sparse/compatibility.py
+++ /dev/null
@@ -1,18 +0,0 @@
-# flake8: noqa F401
-import sys
-
-if sys.version_info < (2, 7):
-    raise ImportError('Need at least Python 2.7.')
-
-if sys.version_info[0] == 3 and sys.version_info[1] < 5:
-    raise ImportError('Need at least Python 3.5 if using Python 3.')
-
-if sys.version_info[0] >= 3:
-    from itertools import zip_longest
-    from builtins import int, range, zip
-else:
-    from itertools import izip_longest as zip_longest
-    from itertools import izip as zip
-
-    int = int
-    range = xrange
diff --git a/sparse/coo/common.py b/sparse/coo/common.py
index 0b2de92..a343bdf 100644
--- a/sparse/coo/common.py
+++ b/sparse/coo/common.py
@@ -1,14 +1,13 @@
 from functools import reduce, wraps
 import operator
 import warnings
-from collections import Iterable
+from collections.abc import Iterable
 
 import numpy as np
 import scipy.sparse
 import numba
 
 from ..sparse_array import SparseArray
-from ..compatibility import range, int
 from ..utils import isscalar, normalize_axis, check_zero_fill_value, check_consistent_fill_value
 
 
@@ -435,7 +434,7 @@ def stack(arrays, axis=0):
     from .core import COO
     check_consistent_fill_value(arrays)
 
-    assert len(set(x.shape for x in arrays)) == 1
+    assert len({x.shape for x in arrays}) == 1
     arrays = [x if isinstance(x, COO) else COO(x) for x in arrays]
     axis = normalize_axis(axis, arrays[0].ndim + 1)
     data = np.concatenate([x.data for x in arrays])
diff --git a/sparse/coo/core.py b/sparse/coo/core.py
index 71accec..dc5bfc1 100644
--- a/sparse/coo/core.py
+++ b/sparse/coo/core.py
@@ -1,6 +1,6 @@
 import copy as _copy
 import operator
-from collections import Iterable, Iterator, Sized
+from collections.abc import Iterable, Iterator, Sized
 from collections import defaultdict, deque
 from functools import reduce
 import warnings
@@ -12,7 +12,6 @@ from numpy.lib.mixins import NDArrayOperatorsMixin
 from .common import dot, matmul
 from .indexing import getitem
 from .umath import elemwise, broadcast_to
-from ..compatibility import int, range
 from ..sparse_array import SparseArray
 from ..utils import normalize_axis, equivalent, check_zero_fill_value, _zero_of_dtype
 
@@ -223,7 +222,7 @@ class COO(SparseArray, NDArrayOperatorsMixin):
             else:
                 shape = ()
 
-        super(COO, self).__init__(shape, fill_value=fill_value)
+        super().__init__(shape, fill_value=fill_value)
         self.coords = self.coords.astype(np.intp)
 
         if self.shape:
@@ -275,7 +274,7 @@ class COO(SparseArray, NDArrayOperatorsMixin):
     def _make_shallow_copy_of(self, other):
         self.coords = other.coords
         self.data = other.data
-        super(COO, self).__init__(other.shape, fill_value=other.fill_value)
+        super().__init__(other.shape, fill_value=other.fill_value)
 
     def enable_caching(self):
         """ Enable caching of reshape, transpose, and tocsr/csc operations
diff --git a/sparse/coo/indexing.py b/sparse/coo/indexing.py
index 7ee9688..12d8399 100644
--- a/sparse/coo/indexing.py
+++ b/sparse/coo/indexing.py
@@ -3,7 +3,8 @@ from numbers import Integral
 import numba
 import numpy as np
 
-from ..compatibility import range, zip_longest
+from itertools import zip_longest
+
 from ..slicing import normalize_index
 from ..utils import _zero_of_dtype, equivalent
 
@@ -582,7 +583,7 @@ def _join_adjacent_pairs(starts_old, stops_old):  # pragma: no cover
     return starts, stops
 
 
-class _AdvIdxInfo(object):
+class _AdvIdxInfo:
     def __init__(self, idx, pos, length):
         self.idx = idx
         self.pos = pos
diff --git a/sparse/coo/umath.py b/sparse/coo/umath.py
index d28c1cb..bfb613b 100644
--- a/sparse/coo/umath.py
+++ b/sparse/coo/umath.py
@@ -4,7 +4,8 @@ import numba
 import numpy as np
 import scipy.sparse
 
-from ..compatibility import range, zip, zip_longest
+from itertools import zip_longest
+
 from ..utils import isscalar, equivalent, _zero_of_dtype
 
 
@@ -379,7 +380,7 @@ def broadcast_to(x, shape):
                sorted=sorted, fill_value=x.fill_value)
 
 
-class _Elemwise(object):
+class _Elemwise:
     def __init__(self, func, *args, **kwargs):
         """
         Initialize the element-wise function calculator.
@@ -481,8 +482,8 @@ class _Elemwise(object):
             fill_value = self.func(*zero_args, **self.kwargs)[()]
 
         if not equivalent(fill_value, fill_value_array).all():
-            raise ValueError('Inconsistent fill-values in the result array: operating on the ndarray with'
-                             ' fill-values produces inconsistent results.')
+            raise ValueError('Performing a mixed sparse-dense operation that would result in a dense array. '
+                             'Please make sure that func(sparse_fill_values, ndarrays) is a constant array.')
 
         # Store dtype separately if needed.
         if self.dtype is not None:
@@ -507,8 +508,8 @@ class _Elemwise(object):
         )
 
         if full_shape != non_ndarray_shape:
-            raise ValueError('All ndarrays must be broadcastable to the shape without ndarrays {}'
-                             .format(non_ndarray_shape))
+            raise ValueError('Please make sure that the broadcast shape of just the sparse arrays is '
+                             'the same as the broadcast shape of all the operands.')
 
         self.shape = full_shape
 
diff --git a/sparse/dok.py b/sparse/dok.py
index 1351cbd..f0b3a3b 100644
--- a/sparse/dok.py
+++ b/sparse/dok.py
@@ -5,7 +5,6 @@ import numpy as np
 from .slicing import normalize_index
 from .utils import equivalent
 from .sparse_array import SparseArray
-from .compatibility import int, range, zip
 
 
 class DOK(SparseArray):
@@ -108,7 +107,7 @@ class DOK(SparseArray):
         if not data:
             data = dict()
 
-        super(DOK, self).__init__(shape, fill_value=fill_value)
+        super().__init__(shape, fill_value=fill_value)
 
         if isinstance(data, dict):
             if not dtype:
@@ -125,7 +124,7 @@ class DOK(SparseArray):
     def _make_shallow_copy_of(self, other):
         self.dtype = other.dtype
         self.data = other.data
-        super(DOK, self).__init__(other.shape, fill_value=other.fill_value)
+        super().__init__(other.shape, fill_value=other.fill_value)
 
     @classmethod
     def from_coo(cls, x):
diff --git a/sparse/slicing.py b/sparse/slicing.py
index 087bd52..e445a4b 100644
--- a/sparse/slicing.py
+++ b/sparse/slicing.py
@@ -2,8 +2,8 @@
 # See license at https://github.com/dask/dask/blob/master/LICENSE.txt
 
 import math
+from collections.abc import Iterable
 from numbers import Integral, Number
-from collections import Iterable
 
 import numpy as np
 
diff --git a/sparse/sparse_array.py b/sparse/sparse_array.py
index 599d289..1a5f4d5 100644
--- a/sparse/sparse_array.py
+++ b/sparse/sparse_array.py
@@ -1,6 +1,5 @@
-from __future__ import absolute_import, division, print_function
 from abc import ABCMeta, abstractmethod
-from collections import Iterable
+from collections.abc import Iterable
 from numbers import Integral
 from functools import reduce
 import operator
@@ -8,10 +7,9 @@ import operator
 import numpy as np
 
 from .utils import _zero_of_dtype
-from .compatibility import int
 
 
-class SparseArray(object):
+class SparseArray:
     """
     An abstract base class for all the sparse array classes.
 
diff --git a/sparse/utils.py b/sparse/utils.py
index 388c4ef..b825fce 100644
--- a/sparse/utils.py
+++ b/sparse/utils.py
@@ -1,5 +1,5 @@
 import functools
-from collections import Iterable
+from collections.abc import Iterable
 from numbers import Integral
 
 import numpy as np
diff --git a/tox.ini b/tox.ini
index 60090f6..ef51e0d 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,8 +1,8 @@
 [tox]
-envlist = py27,py36
+envlist = py36, py37
 [testenv]
 commands=
-    py.test {posargs}
+    pytest {posargs}
 extras=
     tests
     tox


Test patch:
diff --git a/sparse/tests/test_coo.py b/sparse/tests/test_coo.py
index c9699a4..357f7e7 100644
--- a/sparse/tests/test_coo.py
+++ b/sparse/tests/test_coo.py
@@ -1382,7 +1382,6 @@ def test_create_with_lists_of_tuples():
 
 
 def test_sizeof():
-    import sys
     x = np.eye(100)
     y = COO.from_numpy(x)
 
@@ -1832,7 +1831,7 @@ def test_prod_along_axis():
     assert_eq(s2.prod(axis=0), x2.prod(axis=0))
 
 
-class TestRoll(object):
+class TestRoll:
 
     # test on 1d array #
     @pytest.mark.parametrize('shift', [0, 2, -2, 20, -20])
@@ -1920,7 +1919,7 @@ def test_clip():
     assert_eq(out, x.clip(min=1, max=3))
 
 
-class TestFailFillValue(object):
+class TestFailFillValue:
     # Check failed fill_value op
     def test_nonzero_fv(self):
         xs = sparse.random((2, 3), density=0.5, fill_value=1)
@@ -2054,7 +2053,6 @@ def test_out_dtype():
         np.positive(a.todense(), out=b.todense(), dtype='float64').dtype
 
 
-@pytest.mark.skipif(sys.version_info[0] == 2, reason='Test won\'t run on Py2.')
 def test_failed_densification():
     import os
     from importlib import reload
@@ -2072,7 +2070,6 @@ def test_failed_densification():
     reload(sparse)
 
 
-@pytest.mark.skipif(sys.version_info[0] == 2, reason='Test won\'t run on Py2.')
 def test_warn_on_too_dense():
     import os
     from importlib import reload
diff --git a/sparse/tests/test_dok.py b/sparse/tests/test_dok.py
index e9d3d4a..1fed490 100644
--- a/sparse/tests/test_dok.py
+++ b/sparse/tests/test_dok.py
@@ -1,7 +1,6 @@
 import pytest
 
 import numpy as np
-import six
 
 import sparse
 from sparse import DOK
@@ -72,7 +71,7 @@ def test_construct(shape, data):
     s = DOK(shape, data)
     x = np.zeros(shape, dtype=s.dtype)
 
-    for c, d in six.iteritems(data):
+    for c, d in data.items():
         x[c] = d
 
     assert_eq(x, s)