Issue 1:
Original ticket http://projects.scipy.org/numpy/ticket/1631 on 2010-10-10 by @nstange, assigned to @charris.

Hi everybody,

for numpy-1.5.0, the testsuite crashes here with an SIGBUS here:

"Test filled w/ flexible dtype ... Bus Error (core dumped)" (numpy-1.5.0/numpy/ma/tests/test_core.py)

and here:

"Test that 'exotic' formats are processed properly ... Bus Error (core dumped)" (numpy-1.5.0/numpy/ma/tests/test_mrecords.py).

The problem is that SPARC processors require alignment, that is 8 byte values have to be aligned at 8 byte memory boundraries.

The good point is: numpy knows that the array is misaligned (verified using gdb) and thus, the fix is easy (see attached diff 02_put_mask_only_on_behaved_arrays.diff): The check for ISCONTINOUS isn't enough in PyArray_PutMask (item_selection.c), also check for ISBEHAVED (= ISALIGNED && ISWRITABLE).

But now, another problem arises:

a SIGSEGV here:

"Tests fields retrieval"

(numpy-1.5.0/numpy/ma/tests/test_mrecords.py:77).

The problem is in _copy_same_shape(numpy-1.5.0/numpy/core/src/multiarrayctors.c:732): dest->dimensions == NULL.

Please also note that maxaxis == -1 at that point.

Check the attached diff "04_copy_from_same_shape_zerodim_fix.diff" for details. There's one point in the diff I'm unsure about: The right position of

PyArray_INCREF(src);

PyArray_XDECREF(dest);

I tried to resemble the original "logical" position, but since I have no clue about Python's reference counting, please have a look.

While debugging the last SIGSEGV-issue, I stumbled over another mistake (at least I think so):

In PyArray_IterAllButAxis (numpy-1.5.0/numpy/core/src/multiarray/iterators.c), the minaxis won't be set if the first nonzero stride is the smallest one.

See attached diff (03_fix_iterallbutaxis_minstride_search.diff) for a fix.

The reason why I poste those three issues into one report is that the testsuite still doesn't succeed and thus, I'm unsure if I've broken sth. with my diffs.

What I get now is:

FAIL: Ticket #1897 second test

Traceback (most recent call last):

File "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit

e-packages/numpy/core/tests/test_regression.py", line 1255, in test_structured_a

rrays_with_objects2

assert sys.getrefcount(strb) == numb

AssertionError:

7 = <module 'sys' (built-in)>.getrefcount('aaaa')

array([[(0L, 'aaaa'), (0L, 'aaaa')]],

dtype=[('f0', '>i8'), ('f1', '|O4')]) = <module 'numpy' from '/pf/m/m2

22086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/site-packages/numpy

/init.pyc'>.array([[(0,'aaaa'),(1,'bbbb')]], 'i8,O')

array([[(0L, 'aaaa'), (0L, 'aaaa')]],

dtype=[('f0', '>i8'), ('f1', '|O4')])[array([[(0L, 'aaaa'), (0L, 'aaaa

')]],

dtype=[('f0', '>i8'), ('f1', '|O4')]).nonzero()] = array([[(0L, 'aaaa'

), (0L, 'aaaa')]],

dtype=[('f0', '>i8'), ('f1', '|O4')]).ravel()[:1]

assert <module 'sys' (built-in)>.getrefcount('bbbb') == 7

assert <module 'sys' (built-in)>.getrefcount('aaaa') == 7 + 2

FAIL: Test filled w/ mvoid

Traceback (most recent call last):

File "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit

e-packages/numpy/ma/tests/test_core.py", line 518, in test_filled_w_mvoid

assert_equal(tuple(test), (1, default_fill_value(1.)))

File "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit

e-packages/numpy/ma/testutils.py", line 94, in assert_equal

return _assert_equal_on_sequences(actual, desired, err_msg='')

File "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit

e-packages/numpy/ma/testutils.py", line 66, in _assert_equal_on_sequences

assert_equal(actual[k], desired[k], 'item=%r\n%s' % (k,err_msg))

File "/pf/m/m222086/xas/solaris10/python2/python-2.7-ve0-gcc/lib/python2.7/sit

e-packages/numpy/ma/testutils.py", line 98, in assert_equal

raise AssertionError(msg)

AssertionError:

Items are not equal:

item=1

ACTUAL: 2.0

DESIRED: 1e+20

raise AssertionError('\nItems are not equal:\nitem=1\n\n ACTUAL: 2.0\n DESIR

ED: 1e+20')

At least not segfaults/bus errors anymore ;).

Btw.: I don't know what mvoid is, but have a look at the following:

myuid@myhost:~$ ~/xas/solaris10/python2/python-2.7-ve0-gcc/bin/python Python 2.7 (r27:82500, Oct 9 2010, 17:26:38)

[GCC 4.5.1] on sunos5

Type "help", "copyright", "credits" or "license" for more information.

import numpy as np

import numpy.ma as ma

import numpy.ma.core

from numpy.ma.core import *

x = ma.array([(1,2.)], mask=[(0,1)], dtype=[('a', int), ('b', float)])

print x.filled()

[(1, 1e+20)]

x = mvoid((1,2.), mask=[(0,1)], dtype=[('a', int), ('b', float)])

print x.filled()

(1, 2.0)

Since the non-working mvoid-mask could have an impact on results, I cannot release that build to our the scientists at our site. Do you have any idea what the issue could be?

Do you know what a mvoid is? I can't find any documentation about it. I'm seriously thinking about just removing that class from my numpy build (If I knew that there weren't any dependants outside of numpy)...

Wishes

Nicolai

Issue 2:
Original ticket http://projects.scipy.org/numpy/ticket/1094 on 2009-04-24 by trac user Gerard, assigned to @charris.

Running the numpy (version 1.3.0, but also occurs on 1.2.1) autotester fails on masked array test case with a bus error. Used system is Solaris 5.8:

uname -a

SunOS wsasd529 5.8 Generic_117350-18 sun4u sparc SUNW,Sun-Blade-1500

output of "numpy.test(verbose=10)":

...

Tests how masks are initialized at the creation of Maskedarrays. ... ok

Check the use of ndmin ... ok

Make sure we're not losing the original mask w/ ndmin ... ok

Tests creaating a masked array from alist of masked arrays. ... ok

test_deepcopy (test_core.TestMaskedArray) ... ok

Test printing a masked array w/ fancy dtype. ... ok

Test filled w/ flexible dtype ... Bus error

Bus error is caused by a misaligned memory write (see output of purify below):

MAW: Misaligned memory write
This is occurring while in:
DOUBLE_fastputmask [arraytypes.inc.src:2282]
s_val = *vals;
for (i = 0; i < ni; i++) {
if (mask[i]) {
=> in[i] = s_val;
}
}
}
PyArray_PutMask [multiarraymodule.c:4687]
}
}
else {
=> func(dest, mask->data, ni, values->data, nv);
}
}

array_putmask [multiarraymodule.c:4600]
&array, &mask, &values)) {
return NULL;
}
=> return PyArray_PutMask((PyArrayObject *)array, values, mask);
}

/*NUMPY_API
PyCFunction_Call [methodobject.c:77]
PyEval_EvalFrameEx [ceval.c:3564]
PyEval_EvalFrameEx [ceval.c:3650]
PyEval_EvalCodeEx [ceval.c:2831]
PyEval_EvalFrameEx [ceval.c:3659]
...

Commenting the execution of the fastputmask in multiarraymodule.c (and thus running the 'slower' version already implemented there) solves the problem:

multiarraymodule.c, line 4676 (and further):

func = self->descr->f->fastputmask;
/*if (func == NULL) {*/
for (i = 0; i < ni; i++) {
tmp = ((Bool *)(mask->data))[i];
if (tmp) {
src = values->data + chunk*(i % nv);
memmove(dest + i*chunk, src, chunk);
}
}
/*}
else {
func(dest, mask->data, ni, values->data, nv);
}*/

added here are the comment blocks such that func is not executed anymore.

'Minimal' piece of python code to reproduce:

from numpy.ma.core import *

f = array([(1,1)], dtype=[('i',int),('f',float)])

f.mask = [1,1]

f.filled()