Repository: dask/dask

Issue:
Is Masked-array arithmetic getting masks wrong ?
Arithmetic operations between a dask array and a numpy array have the effect of ignoring any mask on the latter.

Like this :
```
>>> lazy = da.asarray([5, 6])
>>> mask = np.ma.masked_array([2, 3], mask=[0, 1])
>>> (lazy + mask).compute()
array([7, 9])
>>> 
>>> # But expected ...
... lazy.compute() + mask
masked_array(data=[7, --],
             mask=[False,  True],
       fill_value=999999)
>>> 
```

However, if we *explicitly* wrap the masked array as a dask array, all is well ...
```
>>> (lazy + da.asanyarray(mask)).compute()
masked_array(data=[1, --],
             mask=[False,  True],
       fill_value=999999)
>>> 
```

(BUT: only using `da.asanyarray` : `da.asarray` does not work)



Discussion:
NOTE:
I think it may be that, when promoting 'mask' to a dask array, 
the code is wrapping it with `da.asarray()`, whereas `da.asanyarray()` would make more sense ?

Or possibly, the fact that `ds.asarray` applies `np.asarray` is ***itself*** a subtle mistake, and this should be calling `np.asanyarray` instead ?
At least, for this precise example, I found that the key call is this one : https://github.com/dask/dask/blob/master/dask/array/core.py#L82
 -- changing that from `np.asarray` to `np.asanyarray` "fixes" the above problem.

However, I'm unclear on the exact intentions of these routines : 
*what is appropriate here ?*
NOTE#2 :

Another thing I found, which may influence our view of this, is that whereas calculating (dask-array + numpy-array) behaves as expected, the opposite order (numpy-array + dask-array) results in realising the dask array and converting it to numpy.

Presumably, this asymmetry is governed by the fact that numpy "accepts" a dask-array as an array-like to be converted, and there is nothing we can do about that in this project.

However, this might influence whether we think this kind of cross-domain arithmetic is desirable in the first place.

@pp-mo thank you for investigating these issues.  

> what is appropriate here ?

The right person to engage here is probably @jcrist , though he may also be busy with other work at the moment.  It may be that you (and the folks around you) are the people best suited to analyze the situation.  
It looks like `elemwise` is calling `da.asarray` on non-dask arrays when it should *possibly* be calling `da.asanyarray`.

> Presumably, this asymmetry is governed by the fact that numpy "accepts" a dask-array as an array-like to be converted, and there is nothing we can do about that in this project.

This is correct, and one reason why I'd advise explicitly converting numpy arrays into dask arrays before mixing them. Things can work if you're careful, but you have greater assurances and control if you handle things manually.
Just reviewing this again...

> I'd advise explicitly converting numpy arrays into dask arrays before mixing them

I get the point, but shouldn't at least `da.asarray` then do the right thing with a masked array ?
The same applies to `da.from_array` : in both cases, you **_must_**  explicitly specify `asarray=False` to avoid it stripping the mask, and again I think this is due to an `np.asarray` which really ought to be a `np.asanyarray`.

If there's any appetite, I'll create a PR suggesting a fix.

@jcrist do you have a recommendation here?  It looks like @pp-mo is willing to do some work here, but needs the go-ahead from you.

Resolving patch:
diff --git a/dask/array/core.py b/dask/array/core.py
--- a/dask/array/core.py
+++ b/dask/array/core.py
@@ -2428,7 +2428,7 @@ def unify_chunks(*args, **kwargs):
     if not args:
         return {}, []
 
-    arginds = [(asarray(a) if ind is not None else a, ind)
+    arginds = [(asanyarray(a) if ind is not None else a, ind)
                for a, ind in partition(2, args)]  # [x, ij, y, jk]
     args = list(concat(arginds))  # [(x, ij), (y, jk)]
     warn = kwargs.get('warn', True)
@@ -2934,19 +2934,12 @@ def asarray(a, **kwargs):
     >>> da.asarray(y)
     dask.array<array, shape=(2, 3), dtype=int64, chunksize=(2, 3)>
     """
-    def frame_types():
-        try:
-            import dask.dataframe as dd
-            return (dd.Series, dd.DataFrame)
-        except ImportError:
-            return ()
-
     if isinstance(a, Array):
         return a
-    if isinstance(a, (list, tuple)) and any(isinstance(i, Array) for i in a):
-        a = stack(a)
-    elif isinstance(a, frame_types()):
+    elif hasattr(a, 'to_dask_array'):
         return a.to_dask_array()
+    elif isinstance(a, (list, tuple)) and any(isinstance(i, Array) for i in a):
+        a = stack(a)
     elif not isinstance(getattr(a, 'shape', None), Iterable):
         a = np.asarray(a)
     return from_array(a, chunks=a.shape, getitem=getter_inline, **kwargs)


Test patch:
diff --git a/dask/array/tests/test_masked.py b/dask/array/tests/test_masked.py
--- a/dask/array/tests/test_masked.py
+++ b/dask/array/tests/test_masked.py
@@ -348,3 +348,17 @@ def test_average_weights_with_masked_array():
     da_avg = da.ma.average(d_a, weights=d_weights, axis=1)
 
     assert_eq(np_avg, da_avg)
+
+
+def test_arithmetic_results_in_masked():
+    mask = np.array([[True, False],
+                     [True, True],
+                     [False, True]])
+    x = np.arange(6).reshape((3, 2))
+    masked = np.ma.array(x, mask=mask)
+    dx = da.from_array(x, chunks=(2, 2))
+
+    res = dx + masked
+    sol = x + masked
+    assert_eq(res, sol)
+    assert isinstance(res.compute(), np.ma.masked_array)