Repository: numpy/numpy

Issue:
Accessing fields for a masked structured array fails with ValueError
I'm trying to create a mask for a structured array. When I try to access a field from the array or simply represent it, numpy throws a `ValueError: field names A not found`, as illustrated below. 

```
>>> R = numpy.empty(10, dtype=[("A", "<f2"), ("B", "<f4")])
>>> Rm = numpy.ma.masked_where(R["A"]<0.1, R)
>>> Rm["A"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 3014, in __getitem__
    dout._mask = _mask[indx]
ValueError: field named A not found.
>>> print(Rm)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 3583, in __str__
    _recursive_printoption(res, m, f)
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 2294, in _recursive_printoption
    (curdata, curmask) = (result[name], mask[name])
ValueError: field named A not found.
>>> print(numpy.version.version)
1.8.0.dev-b8bfcd0
>>> print(numpy.version.git_revision)
b8bfcd02a2f246a9c23675e1650c3d316d733306
```

I tested it with the stable version `1.6.2` and the bleeding-edge version obtained directly from `git`.
It also fails in earlier stable versions (tested with 1.6.2).

Note: if I instead create directly a masked array with `numpy.ma.empty`, I get a _different_ error:

```
>>> R2 = numpy.ma.empty(10, dtype=[("A", "<f2"), ("B", "<f4")])
>>> Rm2 = numpy.ma.masked_where(R2["A"]<0.1, R2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 1810, in masked_where
    cond = mask_or(cond, a._mask)
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 1627, in mask_or
    raise ValueError("Incompatible dtypes '%s'<>'%s'" % (dtype1, dtype2))
ValueError: Incompatible dtypes 'bool'<>'[('A', '?'), ('B', '?')]
```



Discussion:
I don't know if this is the same issue, but with newer numpy versions, the error message has changed:

```
>>> import sys, os
>>> print(numpy.version.version, numpy.version.git_revision, sys.version, os.uname())
1.9.0.dev-a0cf183 a0cf18394d5ce33514fdc37093bd2f65ad4b0dde 3.4.0 (default, Apr  3 2014, 09:15:09) 
[GCC 4.8.1] posix.uname_result(sysname='Linux', nodename='ostrovnoy', release='3.11.0-19-generic', version='#33-Ubuntu SMP Tue Mar 11 18:48:34 UTC 2014', machine='x86_64')
>>> A = numpy.ma.masked_all(shape=5, dtype=[("A", "f4", 3)])
>>> b = A[0]
>>> b["A"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/gerrit/venv/python-3.4-bleed/lib/python3.4/site-packages/numpy/ma/core.py", line 5619, in __getitem__
    if m is not nomask and m[indx]:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```

Still present in version 1.10.0.dev+00ee332 but with a yet different error message:

```
In [131]: R = numpy.empty(10, dtype=[("A", "<f2"), ("B", "<f4")])

In [132]: Rm = numpy.ma.masked_where(R["A"]<0.1, R)

In [133]: Rm["A"]
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-133-7f4318176b14> in <module>()
----> 1 Rm["A"]

/export/data/home/gholl/venv/bleeding/lib/python3.4/site-packages/numpy/ma/core.py in __getitem__(self, indx)
   3071             # Update the mask if needed
   3072             if _mask is not nomask:
-> 3073                 dout._mask = _mask[indx]
   3074                 dout._sharedmask = True
   3075 #               Note: Don't try to check for m.any(), that'll take too long...

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
```

I started to fix this, but as I realised the problem is caused by `Rm.mask.dtype` being unstructured (type `bool`), although `Rm.dtype` is structured.  Is this supposed to be possible?  I asked at Stack Overflow: http://stackoverflow.com/q/28182408/974555


Resolving patch:
diff --git a/numpy/ma/core.py b/numpy/ma/core.py
--- a/numpy/ma/core.py
+++ b/numpy/ma/core.py
@@ -1818,7 +1818,8 @@ def masked_where(condition, a, copy=True):
     else:
         cls = MaskedArray
     result = a.view(cls)
-    result._mask = cond
+    # Assign to *.mask so that structured masks are handled correctly.
+    result.mask = cond
     return result
 
 


Test patch:
diff --git a/numpy/ma/tests/test_core.py b/numpy/ma/tests/test_core.py
--- a/numpy/ma/tests/test_core.py
+++ b/numpy/ma/tests/test_core.py
@@ -3242,6 +3242,15 @@ def test_masked_where_shape_constraint(self):
         test = masked_equal(a, 1)
         assert_equal(test.mask, [0, 1, 0, 0, 0, 0, 0, 0, 0, 0])
 
+    def test_masked_where_structured(self):
+        # test that masked_where on a structured array sets a structured
+        # mask (see issue #2972)
+        a = np.zeros(10, dtype=[("A", "<f2"), ("B", "<f4")])
+        am = np.ma.masked_where(a["A"]<5, a)
+        assert_equal(am.mask.dtype.names, am.dtype.names)
+        assert_equal(am["A"],
+                    np.ma.masked_array(np.zeros(10), np.ones(10)))
+
     def test_masked_otherfunctions(self):
         assert_equal(masked_inside(list(range(5)), 1, 3),
                      [0, 199, 199, 199, 4])