Repository: mne-tools/mne-python

Issue:
Bug in show_fiff for *-trans.fif file?
This looks like a bug in `show_fiff`

``` bash
ISRWKAVW2269D /Applications/MNE-2.7.3-3268-MacOSX-i386/share/mne/mne_analyze/fsaverage$ mne show_fiff fsaverage-trans.fif 
Traceback (most recent call last):
  File "/Users/christian/Code/mne-python/mne/commands/mne_show_fiff.py", line 27, in <module>
    run()
  File "/Users/christian/Code/mne-python/mne/commands/mne_show_fiff.py", line 22, in run
    print(mne.io.show_fiff(args[0]))
  File "/Users/christian/Code/mne-python/mne/io/open.py", line 183, in show_fiff
    out = _show_tree(fid, tree['children'][0], indent=indent, level=0,
IndexError: list index out of range
```



Discussion:
Yeah that's probably a bug, `show_fiff` shouldn't error out if the file exists. Can you make a PR?

Yes I'll look into it

This occurs only for some *-trans.fiff files and not others. What seems to be the reason is that for some trans files there are no "children" (hence the error because `tree['children']` is `[]`). I don't know enough about FIFF files to know what to do, but it seems to me that the "directory" at the root level can also contain information?

This is one that works:

``` py
In [1]: from mne.io.open import fiff_open

In [2]: fname = 'test-trans.fif'

In [3]: f, tree, directory = fiff_open(fname)

In [5]: tree
Out[5]: 
{'block': 0,
 'children': [{'block': array([350], dtype=int32),
   'children': [{'block': array([107], dtype=int32),
     'children': [],
     'directory': [kind: 3506 - type: 3 - size: 4 - next: 0 - pos: 236 - data: None,
      kind: 213 - type: 33 - size: 20 - next: 0 - pos: 256 - data: None,
      kind: 213 - type: 33 - size: 20 - next: 0 - pos: 292 - data: None,
      kind: 213 - type: 33 - size: 20 - next: 0 - pos: 328 - data: None],
     'id': None,
     'nchild': 0,
     'nent': 4,
     'parent_id': None}],
   'directory': [kind: 222 - type: 35 - size: 104 - next: 0 - pos: 96 - data: None],
   'id': None,
   'nchild': 1,
   'nent': 1,
   'parent_id': None}],
 'directory': [kind: 100 - type: 31 - size: 20 - next: 0 - pos: 0 - data: None,
  kind: 101 - type: 3 - size: 4 - next: 0 - pos: 36 - data: None,
  kind: 106 - type: 3 - size: 4 - next: 0 - pos: 56 - data: None,
  kind: 108 - type: 0 - size: 0 - next: -1 - pos: 404 - data: None],
 'id': {'machid': array([-1035262720,        6388], dtype=int32),
  'secs': 1316194493,
  'usecs': 320003,
  'version': 65539},
 'nchild': 1,
 'nent': 4,
 'parent_id': None}

In [6]: import mne

In [7]: mne.read_trans(fname)
Out[7]: 
<Transform  |  head->MRI (surface RAS)>
[[ 0.99930954  0.01275934  0.0348942   0.00206991]
 [ 0.00998479  0.81240475 -0.58300853  0.01130214]
 [-0.03578702  0.58295429  0.81171638 -0.02755522]
 [ 0.          0.          0.          1.        ]]
```

And this is one that does not:

``` py
In [3]: fname = 'fsaverage-trans.fif'

In [4]: f, tree, directory = fiff_open(fname)

In [6]: tree
Out[6]: 
{'block': 0,
 'children': [],
 'directory': [kind: 100 - type: 31 - size: 20 - next: 0 - pos: 0 - data: None,
  kind: 101 - type: 3 - size: 4 - next: 0 - pos: 36 - data: None,
  kind: 106 - type: 3 - size: 4 - next: 0 - pos: 56 - data: None,
  kind: 222 - type: 35 - size: 104 - next: 0 - pos: 76 - data: None,
  kind: 108 - type: 0 - size: 0 - next: -1 - pos: 196 - data: None],
 'id': {'machid': array([39770, 31849], dtype=int32),
  'secs': 1205203475,
  'usecs': 0,
  'version': 65537},
 'nchild': 0,
 'nent': 5,
 'parent_id': None}

In [8]: mne.read_trans(fname)
Out[8]: 
<Transform  |  head->MRI (surface RAS)>
[[  9.99993801e-01  -3.55761126e-03  -5.70168481e-07   1.87336397e-03]
 [  3.55187245e-03   9.98389065e-01  -5.66265360e-02  -2.87957620e-02]
 [  2.02039679e-04   5.66259325e-02   9.98395383e-01  -4.12941016e-02]
 [  0.00000000e+00   0.00000000e+00   0.00000000e+00   1.00000000e+00]]

In [19]: show_fiff(fname)
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-19-e3f0753a20a4> in <module>()
----> 1 show_fiff(fname)

//anaconda/lib/python2.7/site-packages/mne/io/open.pyc in show_fiff(fname, indent, read_limit, max_str, output, verbose)
    181     f, tree, directory = fiff_open(fname)
    182     with f as fid:
--> 183         out = _show_tree(fid, tree['children'][0], indent=indent, level=0,
    184                          read_limit=read_limit, max_str=max_str)
    185     if output == str:

IndexError: list index out of range
```

``` py
In [10]: FIFF.FIFF_COORD_TRANS
Out[10]: 222
```

and 222 is in the root directory of the second, erroring file....

If I modify show_fiff to start with the `tree` instead of `tree['children'][0]` I get 

``` bash
$ mne show_fiff test-trans.fif 
0 = ???
    100 = FIFF_FILE_ID (20) = {'secs': 1316194493, 'version' ... dict len=4
    101 = FIFF_DIR_POINTER (4) = [-1]
    106 = FIFF_FREE_LIST (4) = [-1]
    108 = FIFF_NOP (0)
    [350] = FIFFB_MNE
        222 = FIFF_COORD_TRANS (104) = <Transform  |  head->MRI (surf ... dict len=3
        [107] = FIFFB_ISOTRAK
            3506 = FIFF_MNE_COORD_FRAME (4) = [5]
            x3: 213 = FIFF_DIG_POINT (20) = {'coord_frame': 0, 'ident': 3, ... dict len=4
```

and 

``` bash
$ mne show_fiff fsaverage-trans.fif 
0 = ???
    100 = FIFF_FILE_ID (20) = {'secs': 1205203475, 'version' ... dict len=4
    101 = FIFF_DIR_POINTER (4) = [-1]
    106 = FIFF_FREE_LIST (4) = [-1]
    222 = FIFF_COORD_TRANS (104) = <Transform  |  head->MRI (surf ... dict len=3
    108 = FIFF_NOP (0)
```

Thanks for the investigation, I think I found a reasonable fix


Resolving patch:
diff --git a/mne/commands/mne_show_fiff.py b/mne/commands/mne_show_fiff.py
--- a/mne/commands/mne_show_fiff.py
+++ b/mne/commands/mne_show_fiff.py
@@ -8,6 +8,7 @@
 
 # Authors : Eric Larson, PhD
 
+import codecs
 import sys
 import mne
 
@@ -19,6 +20,11 @@ def run():
     if len(args) != 1:
         parser.print_help()
         sys.exit(1)
+    # This works around an annoying bug on Windows for show_fiff, see:
+    # https://pythonhosted.org/kitchen/unicode-frustrations.html
+    if int(sys.version[0]) < 3:
+        UTF8Writer = codecs.getwriter('utf8')
+        sys.stdout = UTF8Writer(sys.stdout)
     print(mne.io.show_fiff(args[0]))
 
 
diff --git a/mne/datasets/utils.py b/mne/datasets/utils.py
--- a/mne/datasets/utils.py
+++ b/mne/datasets/utils.py
@@ -167,7 +167,7 @@ def _data_path(path=None, force_update=False, update_path=True, download=True,
     path = _get_path(path, key, name)
     # To update the testing or misc dataset, push commits, then make a new
     # release on GitHub. Then update the "releases" variable:
-    releases = dict(testing='0.20', misc='0.1')
+    releases = dict(testing='0.23', misc='0.1')
     # And also update the "hashes['testing']" variable below.
 
     # To update any other dataset, update the data archive itself (upload
@@ -211,7 +211,7 @@ def _data_path(path=None, force_update=False, update_path=True, download=True,
         sample='1d5da3a809fded1ef5734444ab5bf857',
         somato='f3e3a8441477bb5bacae1d0c6e0964fb',
         spm='f61041e3f3f2ba0def8a2ca71592cc41',
-        testing='625cb9c1c62de03bd2bc6a3ca9a6f2d8'
+        testing='7ddb41eef97bb2ae6bdb371dc3b5d53d',
     )
     folder_origs = dict(  # not listed means None
         misc='mne-misc-data-%s' % releases['misc'],
diff --git a/mne/io/open.py b/mne/io/open.py
--- a/mne/io/open.py
+++ b/mne/io/open.py
@@ -179,8 +179,11 @@ def show_fiff(fname, indent='    ', read_limit=np.inf, max_str=30,
     if output not in [list, str]:
         raise ValueError('output must be list or str')
     f, tree, directory = fiff_open(fname)
+    # This gets set to 0 (unknown) by fiff_open, but FIFFB_ROOT probably
+    # makes more sense for display
+    tree['block'] = FIFF.FIFFB_ROOT
     with f as fid:
-        out = _show_tree(fid, tree['children'][0], indent=indent, level=0,
+        out = _show_tree(fid, tree, indent=indent, level=0,
                          read_limit=read_limit, max_str=max_str)
     if output == str:
         out = '\n'.join(out)
@@ -189,6 +192,7 @@ def show_fiff(fname, indent='    ', read_limit=np.inf, max_str=30,
 
 def _find_type(value, fmts=['FIFF_'], exclude=['FIFF_UNIT']):
     """Helper to find matching values"""
+    value = int(value)
     vals = [k for k, v in six.iteritems(FIFF)
             if v == value and any(fmt in k for fmt in fmts) and
             not any(exc in k for exc in exclude)]
@@ -203,7 +207,7 @@ def _show_tree(fid, tree, indent, level, read_limit, max_str):
     this_idt = indent * level
     next_idt = indent * (level + 1)
     # print block-level information
-    out = [this_idt + str(tree['block'][0]) + ' = ' +
+    out = [this_idt + str(int(tree['block'])) + ' = ' +
            '/'.join(_find_type(tree['block'], fmts=['FIFFB_']))]
     if tree['directory'] is not None:
         kinds = [ent.kind for ent in tree['directory']] + [-1]


Test patch:
diff --git a/mne/tests/test_dipole.py b/mne/tests/test_dipole.py
--- a/mne/tests/test_dipole.py
+++ b/mne/tests/test_dipole.py
@@ -39,7 +39,7 @@
                       'sample_audvis_trunc-trans.fif')
 fname_fwd = op.join(data_path, 'MEG', 'sample',
                     'sample_audvis_trunc-meg-eeg-oct-6-fwd.fif')
-fname_xfit_dip = op.join(data_path, 'misc', 'fam_115_LH.fif')
+fname_xfit_dip = op.join(data_path, 'dip', 'fixed_auto.fif')
 subjects_dir = op.join(data_path, 'subjects')
 
 
diff --git a/mne/tests/test_utils.py b/mne/tests/test_utils.py
--- a/mne/tests/test_utils.py
+++ b/mne/tests/test_utils.py
@@ -8,6 +8,7 @@
 import warnings
 
 from mne import read_evokeds
+from mne.datasets import testing
 from mne.externals.six.moves import StringIO
 from mne.io import show_fiff
 from mne.utils import (set_log_level, set_log_file, _TempDir,
@@ -32,6 +33,10 @@
 fname_log = op.join(base_dir, 'test-ave.log')
 fname_log_2 = op.join(base_dir, 'test-ave-2.log')
 
+data_path = testing.data_path(download=False)
+fname_fsaverage_trans = op.join(data_path, 'subjects', 'fsaverage', 'bem',
+                                'fsaverage-trans.fif')
+
 
 def clean_lines(lines=[]):
     # Function to scrub filenames for checking logging output (in test_logging)
@@ -365,6 +370,7 @@ def test_config():
         assert_raises(RuntimeError, set_config, key, 'true', home_dir=tempdir)
 
 
+@testing.requires_testing_data
 def test_show_fiff():
     """Test show_fiff
     """
@@ -375,6 +381,7 @@ def test_show_fiff():
             'FIFF_EPOCH']
     assert_true(all(key in info for key in keys))
     info = show_fiff(fname_raw, read_limit=1024)
+    assert_true('COORD_TRANS' in show_fiff(fname_fsaverage_trans))
 
 
 @deprecated('message')