Repository: mne-tools/mne-python

Issue:
mne.create_info returns unhashable mne.Info, see meas_info.py:788
minor bug, easy fix
  
    >>> mne.__version__
    '0.16.dev0'

I'm building mne.Raw from scratch with 

     >>> info = mne.create_info(chanlabels, 
                                samplerate,
                                chantypes,
                                montage)
 

The EEG channel labels are not of the form 'EEGNNN' so when when info['dig'] gets built, the *idents* handler falls thru to an np.arange which is dtype('int64')

     # meas_info.py

     ...

     787        except ValueError:  # and if any conversion fails, simply use arange
     788              idents = np.arange(1, len(keys) + 1)
     789        for key, ident in zip(keys, idents):

     ...
uh oh ...

     >>>  mne.utils.object_hash(info)
     *** RuntimeError: unsupported type: <class 'numpy.int64'> (1)


hack-a-patch for testing clears up the exception

     >>> for d in info['dig']:
             d['ident'] = int(d['ident'])
     >>> mne.utils.object_hash(info)
     184875521553997324099927900017502440941

Making *idents* an iterable of ints in meas_info.py:788 should do the trick. 

Let me know if you want me to do the fix.

Thanks.

Tom




Discussion:
I think we do indeed usually cast these to `int` when reading, so doing it when creating from scratch makes sense. PR welcome, it's best if you can follow TDD by adding a tiny test that fails on `master` but passes on your PR.
our hashing code cannot hash array of int64?

Actually this is a bit strange, `mne.utils.object_hash(np.int64(1))` passes on my system. @turbach does it fail for you? What is your `mne.sys_info()`?
looks like np.ndarrays are in the switch but i don't see how np scalar
dtypes are handled when the recursion bottoms out  ... utils.py:157



On Nov 29, 2017 11:29 PM, "Alexandre Gramfort" <notifications@github.com>
wrote:

our hashing code cannot hash array of int64?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<https://github.com/mne-tools/mne-python/issues/4796#issuecomment-348105481>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AGM9yGHrv2fAajR3PuT_DBkjbI0SQOdIks5s7llygaJpZM4Qv3SB>
.

Interesting that mne.utils.object_hash(np.int64(1)) works on your system
... I'm curious what elif clause in utils.py::object_hash is catching it
for you.

I did some digging before submitting the issue, here's what I see ...

(Pdb) mne.utils.object_hash(np.int64(1))
*** RuntimeError: unsupported type: <class 'numpy.int64'> (1)

(Pdb) isinstance(np.int64(1), np.ndarray)
False

(Pdb) isinstance(np.int64(1), int)
False

So on my system np.int64(1) falls through all the elif clauses in
object_hash and the else clause and raise the exception.

I thought this was by design and a stray np scalar had crept into
info['dig'] ... submitted the issue accordingly.

(Pdb) mne.sys_info()
Platform:
Linux-3.10.0-514.6.2.el7.x86_64-x86_64-with-centos-7.3.1611-Core
Python:        3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016,
12:22:00)  [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Executable:    /usr/local/anaconda3/bin/python
CPU:           x86_64: 48 cores
Memory:        503.6 GB

mne:           0.16.dev0
numpy:         1.11.3 {blas=mkl_intel_lp64, lapack=mkl_intel_lp64}
scipy:         0.18.1
matplotlib:    2.0.0

sklearn:       0.18.1
nibabel:       Not found
mayavi:        Not found
pycuda:        2017.1.1
skcuda:        0.5.1
pandas:        0.20.1


On Thu, Nov 30, 2017 at 8:01 AM, Tom Urbach <t.p.urbach@gmail.com> wrote:

> looks like np.ndarrays are in the switch but i don't see how np scalar
> dtypes are handled when the recursion bottoms out  ... utils.py:157
>
>
>
> On Nov 29, 2017 11:29 PM, "Alexandre Gramfort" <notifications@github.com>
> wrote:
>
> our hashing code cannot hash array of int64?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/mne-tools/mne-python/issues/4796#issuecomment-348105481>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AGM9yGHrv2fAajR3PuT_DBkjbI0SQOdIks5s7llygaJpZM4Qv3SB>
> .
>
>
>

I can replicate the pb:

In [1]: import mne

In [2]: import numpy as np

In [3]: mne.utils.object_hash(np.int64(1))
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-3-6b8634000e2a> in <module>()
----> 1 mne.utils.object_hash(np.int64(1))

~/work/src/mne-python/mne/utils.py in object_hash(x, h)
    169             object_hash(xx, h)
    170     else:
--> 171         raise RuntimeError('unsupported type: %s (%s)' % (type(x),
x))
    172     return int(h.hexdigest(), 16)
    173

RuntimeError: unsupported type: <class 'numpy.int64'> (1)


Resolving patch:
diff --git a/mne/utils.py b/mne/utils.py
--- a/mne/utils.py
+++ b/mne/utils.py
@@ -158,7 +158,7 @@ def object_hash(x, h=None):
     elif isinstance(x, (string_types, float, int, type(None))):
         h.update(str(type(x)).encode('utf-8'))
         h.update(str(x).encode('utf-8'))
-    elif isinstance(x, np.ndarray):
+    elif isinstance(x, (np.ndarray, np.number, np.bool_)):
         x = np.asarray(x)
         h.update(str(x.shape).encode('utf-8'))
         h.update(str(x.dtype).encode('utf-8'))


Test patch:
diff --git a/mne/tests/test_utils.py b/mne/tests/test_utils.py
--- a/mne/tests/test_utils.py
+++ b/mne/tests/test_utils.py
@@ -273,6 +273,10 @@ def test_hash():
     y = 0
     assert_true('type mismatch' in object_diff(x, y))
 
+    # smoke test for gh-4796
+    assert object_hash(np.int64(1)) != 0
+    assert object_hash(np.bool_(True)) != 0
+
 
 def test_md5sum():
     """Test md5sum calculation."""