Repository: mne-tools/mne-python

Issue:
Plotting ica properties returns ValueError (dtype 64)
I'm trying to follow the Artifact Correction with ICA example with Python 2.7.14 under Spyder in 64-bit mode (conda version 4.3.29). Everything works as expected until:

**ica.plot_properties(raw, picks=0)**

which leads to an ValueError.


>   File "<ipython-input-33-e9d421a0f5d6>", line 1, in <module>
>     ica.plot_properties(raw,picks=picks)
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\mne\preprocessing\ica.py", line 1435, in plot_properties
>     figsize=figsize, show=show)
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\mne\viz\ica.py", line 241, in plot_ica_properties
>     psds, freqs = psd_multitaper(epochs_src, picks=picks, **psd_args)
> 
>   File "<string>", line 2, in psd_multitaper
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\mne\utils.py", line 728, in verbose
>     return function(*args, **kwargs)
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\psd.py", line 292, in psd_multitaper
>     n_jobs=n_jobs, verbose=verbose)
> 
>   File "<string>", line 2, in psd_array_multitaper
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\mne\utils.py", line 728, in verbose
>     return function(*args, **kwargs)
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 521, in psd_array_multitaper
>     low_bias=low_bias)
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 201, in dpss_windows
>     select_range=(N - Kmax, N - 1))
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 847, in eigvals_banded
>     return eig_banded(a_band, lower=lower, eigvals_only=1,
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 587, in eig_banded
>     raise ValueError('expected two-dimensional array')
> 
>   File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 478, in _check_select
>     if sr.dtype.char.lower() not in 'lih':
> 
> **ValueError: when using select="i", select_range must contain integers, got dtype int64**

I'm quite new to Python, but I guess it has something to do with using the 64-bit version? I tried it with 32-bit mode, but then I ended up with a MemoryError. Does anyone have a suggestion on how to resolve this problem?



Discussion:
This one is a bit weird. Are you on Windows?

Can you change line 509 of `mne/time_frequency/multitaper.py` from:
```
n_times = x.shape[-1]
```
to
```
n_times = int(x.shape[-1])
```
To see if it fixes the problem?
> Are you on Windows?

Yes I am using Windows 7 (64-Bit).

> To see if it fixes the problem?

I changed the line accordingly, but the error still remains the same.
If I skip this step and move on to the artifact detection using EOG information multitaper.py produces another error:

>File "...\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 509, in psd_array_multitaper
    dshape = int(x.shape[:-1])
  TypeError: int() argument must be a string or a number, not 'tuple'
windows does not use standard int64. One fix is certain to do a np.astype
somewhere
but for this you need to debug after putting a breakpoint at the right
place.

what is the value of sr.dtype ?

> what is the value of sr.dtype ?

The value of sr.dtype is int64

Somehow the tuple select_range from multitaper.py (line 201) ends up as a dstype = int64 via np.asarray(select_range)?
no idea

I checked for sr.dtype.char.lower() and it is set to 'q' which is not in 'lih'. When adding q to the string the correct figure would be produced.

If I just create the numpyarray manually and check for dtype.char I get the following:

> sr.astype(np.int64)
Out[21]: array([1], dtype=int64)

> sr.dtype.char.lower()
Out[22]: 'l' 

I have no idea why the type is 'q' in one case, but 'l' in the other.
Me neither, but let's go ahead and add it to the list (it's `c long long` equiv)

https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.scalars.html
... oh wait it's in a SciPy function. You need to cast it in MNE to make it happy, maybe pass `np.int64(N)` or something?
BTW doing `sr.astype(np.int64)` returns something, it does not change `sr` in place
After adding np.int64(N) to line 201 in multitaper.py the value error is solved:

> w = linalg.eigvals_banded(ab, select='i', 
>                                         select_range=(np.int64(N) - Kmax, np.int64(N) - 1))


How can I fix this issue? I am struggling with the same problem!
how do we replicate? can you replicate on sample data?
Yes, both sample data using this script https://martinos.org/mne/stable/auto_examples/preprocessing/plot_run_ica.html#sphx-glr-auto-examples-preprocessing-plot-run-ica-py and on my data give me the same issue mentioned before. I changed those lines suggested by nrosjat in the multitaper.py file, but I keep getting the same value error.
`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-fbaab2c30175> in <module>()
     56 ########################################################################
#######
     57 # Plot properties of ECG components:
---> 58 ica.plot_properties(epochs, picks=ecg_inds)

D:\Users\Davide\Anaconda2\lib\site-packages\mne\preprocessing\ica.pyc in plot_pr
operties(self, inst, picks, axes, dB, plot_std, topomap_args, image_args, psd_ar
gs, figsize, show)
   1433                                    topomap_args=topomap_args,
   1434                                    image_args=image_args, psd_args=psd_a
rgs,
-> 1435                                    figsize=figsize, show=show)
   1436
   1437     @copy_function_doc_to_method_doc(plot_ica_sources)

D:\Users\Davide\Anaconda2\lib\site-packages\mne\viz\ica.pyc in plot_ica_properti
es(ica, inst, picks, axes, dB, plot_std, topomap_args, image_args, psd_args, fig
size, show)
    239     plot_lowpass_edge = inst.info['lowpass'] < Nyquist and (
    240         psd_args['fmax'] > inst.info['lowpass'])
--> 241     psds, freqs = psd_multitaper(epochs_src, picks=picks, **psd_args)
    242
    243     def set_title_and_labels(ax, title, xlab, ylab):

D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\psd.pyc in psd_mu
ltitaper(inst, fmin, fmax, tmin, tmax, bandwidth, adaptive, low_bias, normalizat
ion, picks, proj, n_jobs, verbose)

D:\Users\Davide\Anaconda2\lib\site-packages\mne\utils.pyc in verbose(function, *
args, **kwargs)
    726         with use_log_level(verbose_level):
    727             return function(*args, **kwargs)
--> 728     return function(*args, **kwargs)
    729
    730

D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\psd.pyc in psd_mu
ltitaper(inst, fmin, fmax, tmin, tmax, bandwidth, adaptive, low_bias, normalizat
ion, picks, proj, n_jobs, verbose)
    290                                 bandwidth=bandwidth, adaptive=adaptive,
    291                                 low_bias=low_bias, normalization=normali
zation,
--> 292                                 n_jobs=n_jobs, verbose=verbose)

D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in
 psd_array_multitaper(x, sfreq, fmin, fmax, bandwidth, adaptive, low_bias, norma
lization, n_jobs, verbose)

D:\Users\Davide\Anaconda2\lib\site-packages\mne\utils.pyc in verbose(function, *
args, **kwargs)
    726         with use_log_level(verbose_level):
    727             return function(*args, **kwargs)
--> 728     return function(*args, **kwargs)
    729
    730

D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in
 psd_array_multitaper(x, sfreq, fmin, fmax, bandwidth, adaptive, low_bias, norma
lization, n_jobs, verbose)
    519     n_tapers_max = int(2 * half_nbw)
    520     dpss, eigvals = dpss_windows(n_times, half_nbw, n_tapers_max,
--> 521                                  low_bias=low_bias)
    522
    523     # decide which frequencies to keep

D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in
 dpss_windows(N, half_nbw, Kmax, low_bias, interp_from, interp_kind)
    199         # only calculate the highest Kmax eigenvalues
    200         w = linalg.eigvals_banded(ab, select='i',
--> 201                                   select_range=(np.int64(N) - Kmax, np.i
nt64(N) - 1))
    202         w = w[::-1]
    203

D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in eigvals_b
anded(a_band, lower, overwrite_a_band, select, select_range, check_finite)
    845     return eig_banded(a_band, lower=lower, eigvals_only=1,
    846                       overwrite_a_band=overwrite_a_band, select=select,
--> 847                       select_range=select_range, check_finite=check_fini
te)
    848
    849

D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in eig_bande
d(a_band, lower, eigvals_only, overwrite_a_band, select, select_range, max_ev, c
heck_finite)
    585         raise ValueError('expected two-dimensional array')
    586     select, vl, vu, il, iu, max_ev = _check_select(
--> 587         select, select_range, max_ev, a1.shape[1])
    588     del select_range
    589     if select == 0:

D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in _check_se
lect(select, select_range, max_ev, max_len)
    476             if sr.dtype.char.lower() not in 'lih':
    477                 raise ValueError('when using select="i", select_range mu
st '
--> 478                                  'contain integers, got dtype %s' % sr.d
type)
    479             # translate Python (0 ... N-1) into Fortran (1 ... N) with +
 1
    480             il, iu = sr + 1

ValueError: when using select="i", select_range must contain integers, got dtype
 int64`
This is the error I get when trying to plot ica properties. Do I have another way to reach that data?
I get the same error when trying to run https://www.martinos.org/mne/stable/auto_examples/connectivity/plot_sensor_connectivity.html#sphx-glr-auto-examples-connectivity-plot-sensor-connectivity-py 
what does:

mne.sys_info()

return on your system? what OS are you using? win32 or win64?

In [2]: mne.sys_info()
D:\Users\Davide\Anaconda2\lib\site-packages\h5py\__init__.py:36: FutureWarning:

Conversion of the second argument of issubdtype from `float` to `np.floating` is
 deprecated. In future, it will be treated as `np.float64 == np.dtype(float).typ
e`.

Platform:      Windows-7-6.1.7601-SP1
Python:        2.7.14 |Anaconda, Inc.| (default, Nov  8 2017, 13:40:45) [MSC v.1
500 64 bit (AMD64)]
Executable:    D:\Users\Davide\Anaconda2\python.exe
CPU:           Intel64 Family 6 Model 42 Stepping 7, GenuineIntel: 4 cores
Memory:        7.9 GB

mne:           0.15.2
numpy:         1.14.0 {lapack=mkl_rt, blas=mkl_rt}
scipy:         1.0.0
matplotlib:    2.1.2

sklearn:       0.19.1
nibabel:       2.2.1
mayavi:        4.5.0
pycuda:        Not found
skcuda:        Not found
pandas:        0.22.0

Resolving patch:
diff --git a/mne/time_frequency/multitaper.py b/mne/time_frequency/multitaper.py
--- a/mne/time_frequency/multitaper.py
+++ b/mne/time_frequency/multitaper.py
@@ -3,6 +3,7 @@
 
 # Parts of this code were copied from NiTime http://nipy.sourceforge.net/nitime
 
+import operator
 import numpy as np
 from scipy import linalg
 
@@ -144,7 +145,10 @@ def dpss_windows(N, half_nbw, Kmax, low_bias=True, interp_from=None,
     """
     from scipy import interpolate
     from ..filter import next_fast_len
-    Kmax = int(Kmax)
+    # This np.int32 business works around a weird Windows bug, see
+    # gh-5039 and https://github.com/scipy/scipy/pull/8608
+    Kmax = np.int32(operator.index(Kmax))
+    N = np.int32(operator.index(N))
     W = float(half_nbw) / N
     nidx = np.arange(N, dtype='d')
 


Test patch: