Repository: dask/dask

Issue:
Results are not correctly written to `out` in dask.array functions
Many NumPy functions allow an optional arg/kwarg `out` for results to be written into. When this feature is used in `dask.array`, the result returned by the function is correct but the dask array passed to `out` does not contain correct values.

I've tested and confirmed this for `da.maximum`, `da.minimum`, `da.cos`, and `da.sin` after which I decided it was likely a general problem.

Minimal example (replace the two `result =` calls to test different functions):

``` python
import numpy as np
import dask.array as da

test = da.arange(-128, 128, chunks=(128,))
out = da.empty(256, chunks=(128,))
result = da.minimum(test, 0, out=out)  # result is correct, out is not

nptest = test.compute()
npout = np.empty(256)
npresult = np.minimum(nptest, 0, out=npout)  # stored correctly in out

# Test NumPy first
np.testing.assert_allclose(npresult, npout)  # succeeds
np.testing.assert_allclose(result.compute(), out.compute())  # fails
```

EDIT: NumPy describes the intended behavior of `out` as: "If `out` is provided, the function writes the result into it, and returns a reference to `out`."



Discussion:
Support for `out` is indeed lacking.  It's unfortunate that this didn't err.  Many `dask.array` functions accept `kwargs` and so silently fail in cases like this.

There are two solutions
1.  Support out parameters
2.  Improve handling of keyword arguments

I think that the choice between these two depends on whether or not out-parameters are valuable enough to warrant the extra maintenance costs.

@JDWarner why were you using out parameters?  Was it for computational efficiency or for programming convenience?

Computational efficiency in this case won't be impacted.  Using the `.store` method will store the result as it is computed.

@mrocklin I'll freely acknowledge that `out` is essentially irrelevant for efficiency _when using Dask_.

The reason I ran into this is because I've been making a proof-of-concept showing it's possible to support Dask arrays and NumPy arrays on a shared codebase in scikit-image. Our algorithms not infrequently use `out` to improve efficiency for NumPy, and since the option seemed to be understood I used it to make the compatibility as simple as possible. 

So... convenience. I can handle this in the compatibility layer, but in case it wasn't intended I raised the Issue. 


Resolving patch:
diff --git a/dask/array/core.py b/dask/array/core.py
--- a/dask/array/core.py
+++ b/dask/array/core.py
@@ -1108,7 +1108,7 @@ def normalize_chunks(chunks, shape=None):
     return tuple(map(tuple, chunks))
 
 
-def from_array(x, chunks, name=None, lock=False, **kwargs):
+def from_array(x, chunks, name=None, lock=False):
     """ Create dask array from something that looks like an array
 
     Input must have a ``.shape`` and support numpy-style slicing.
@@ -1238,7 +1238,10 @@ def atop(func, out_ind, *args, **kwargs):
         top - dict formulation of this function, contains most logic
     """
     out = kwargs.pop('name', None) or next(names)
-    dtype = kwargs.get('dtype', None)
+    dtype = kwargs.pop('dtype', None)
+    if kwargs:
+        raise TypeError("%s does not take the following keyword arguments %s" %
+            (func.__name__, str(sorted(kwargs.keys()))))
     arginds = list(partition(2, args)) # [x, ij, y, jk] -> [(x, ij), (y, jk)]
     numblocks = dict([(a.name, a.numblocks) for a, ind in arginds])
     argindsstr = list(concat([(a.name, ind) for a, ind in arginds]))
@@ -1566,6 +1569,9 @@ def elemwise(op, *args, **kwargs):
     --------
     atop
     """
+    if not set(['name', 'dtype']).issuperset(kwargs):
+        raise TypeError("%s does not take the following keyword arguments %s" %
+            (op.__name__, str(sorted(set(kwargs) - set(['name', 'dtype'])))))
     name = kwargs.get('name') or next(names)
     out_ndim = max(len(arg.shape) if isinstance(arg, Array) else 0
                    for arg in args)


Test patch:
diff --git a/dask/array/tests/test_array_core.py b/dask/array/tests/test_array_core.py
--- a/dask/array/tests/test_array_core.py
+++ b/dask/array/tests/test_array_core.py
@@ -1123,3 +1123,12 @@ def test_chunks_is_immutable():
         assert False
     except TypeError as e:
         assert 'rechunk(2)' in str(e)
+
+
+def test_raise_on_bad_kwargs():
+    x = da.ones(5, chunks=3)
+    try:
+        da.minimum(x, out=None)
+    except TypeError as e:
+        assert 'minimum' in str(e)
+        assert 'out' in str(e)