Package: mne-hcp
.. -*- mode: rst -*-|Travis|_ |Zenodo|_ |Codecov|_.. |Travis| image:: https://api.travis-ci.org/mne-tools/mne-hcp.png?branch=master.. _Travis: https://travis-ci.org/mne-tools/mne-hcp.. |Zenodo| image:: https://zenodo.org/badge/53261823.svg.. _Zenodo: https://zenodo.org/badge/latestdoi/53261823.. |Codecov| image:: http://codecov.io/github/mne-tools/mne-hcp/coverage.svg?branch=master.. _Codecov: http://codecov.io/github/mne-tools/mne-hcp?branch=masterMNE-HCP=======We provide Python tools for seamless integration of MEG data from the `Human Connectome Project  <http://www.humanconnectome.org>`_ into the Python ecosystem.In only a few lines of code, complex data retrieval requests can be readily executed on the resources from this neuroimaging reference dataset. Raw HCP data are translated into actionable MNE objects that we know and love. MNE-HCP abstracts away difficulties due to diverging coordinate systems, distributed information, and file format conventions. Providing a simple and consistent access to HCP MEG data will facilitate emergence of standardized data analysis practices.By building on the `MNE software package <http://martinos.org/mne/>`_, MNE-HCP naturally supplements a fast growing stack of Python data science toolkits.Fast interface to MEG data--------------------------Allow us to give you a flavor by a few example queries of MEG HCP data from subject 1003007:.. code-block:: python# Get all entries from the MEG data headerinfo = hcp.read_info('1003007', 'task_motor')# Get continuous MEG time seriesraw = hcp.read_raw('1003007', 'task_motor')# Get segmented MEG time seriesepochs = hcp.read_epochs('1003007', 'task_motor')# Get all MEG time series averaged across eventslist_of_evoked = hcp.read_evokeds('1003007', 'task_motor')# Get details on contamination and noise sourcesannotations_dict = hcp.read_annot('1003007', 'task_motor')# Get precomputed independent components that compose the signal time seriesica_mat = hcp.read_ica('1003007', 'task_motor')Scope and Disclaimer--------------------This code is under active research-driven development. The API is still changing,but is getting closer to a stable release... note::For now please consider the following caveats:- We only intend to support a subset of the files shipped with HCP.- Specifically, for now it is not planned to support io and processing for any outputs of the HCP source space pipelines.- This library breaks with some of MNE conventions in order to make the HCP outputs compatible with MNE.Installation============We recommend the `Anaconda Python distribution <https://www.continuum.io/downloads>`_, which comes with the necessary dependencies. Alternatively, to install ``mne-hcp``, you first need to install its dependencies::$ pip install numpy matplotlib scipy scikit-learn mne joblib pandasThen clone the repository::$ git clone http://github.com/mne-tools/mne-hcpand finally run `setup.py` to install the package::$ cd mne-hcp/$ python setup.py installIf you do not have admin privileges on the computer, use the ``--user`` flagwith `setup.py`.Alternatively, for a devoloper install based on symbolic links (which simplifies keeping up with code changes), do::$ cd mne-hcp/$ python setup.py developTo check if everything worked fine, you can do::$ python -c 'import hcp'and it should not give any error messages.Dependencies------------The following main and additional dependencies are required to use MNE-HCP:- MNE-Python master branch- the MNE-Python dependencies, specifically- scipy- numpy- matplotlib- scikit-learn (optional)Quickstart==========The following data layout is expected: a folder that contains the HCP dataas they are unpacked by a zip, subject wise.When data were downloaded via the Aspera connect client, the followingcommand should produce the expected layout:.. code-block:: bash$ for fname in $(ls *zip); do$    echo unpacking $fname;$    unzip -o $fname; rm $fname;$ doneWhen files are downloaded using the `Amazon webservice tools <http://s3tools.org/s3cmd>`_, e.g. `s3rcmd`,all should be fine.The code is organized by different modules.The `io` module includes readers for sensor space data at different processingstages and annotations for bad data.Types of Data-------------MNE-HCP uses custom names for values that are more MNE-pythonic, the followingtable gives an overview:+-----------------------+-------------------------------------+----------------+| **name**              | **readers**                         | **HCP jargon** |+-----------------------+-------------------------------------+----------------+| 'rest'                | raw, epochs, info, annotations, ica | 'Restin'       |+-----------------------+-------------------------------------+----------------+| 'task_working_memory' | raw, epochs, info, annotations, ica | 'Wrkmem'       |+-----------------------+-------------------------------------+----------------+| 'task_story_math'     | raw, epochs, info, annotations, ica | 'StoryM'       |+-----------------------+-------------------------------------+----------------+| 'task_motor'          | raw, epochs, info, annotations, ica | 'Motor'        |+-----------------------+-------------------------------------+----------------+| 'noise_subject'       | raw, info                           | 'Pnoise'       |+-----------------------+-------------------------------------+----------------+| 'noise_empty_room'    | raw, info                           | 'Rnoise'       |+-----------------------+-------------------------------------+----------------+Functionality to make the HCP datasets compatible with MNE----------------------------------------------------------MNE HCP comes with convenience functions such as `hcp.make_mne_anatomy`. This one will create anMNE friendly anatomy directories and extracts the head model andcoregistration MEG to MRI coregistration.(Yes, it maps to MRI, not to the helmet -- a peculiarity of the HCP data.)It can be used as follows:.. code-block:: python>>> import os.path as op>>> import hcp>>> storage_dir = op.expanduser('~/data/MNE-HCP')>>> hcp.make_mne_anatomy(>>>     '100307', subjects_dir=storage_dir + '/subjects',>>>     hcp_path=storage_dir + '/HCP',>>>     recordings_path=storage_dir + '/hcp-meg')reading extended structural processing ...reading RAS freesurfer transformCombining RAS transform and coregistrationextracting head modelcoregistring head model to MNE-HCP coordinatesextracting coregistrationFile Mapping------------MNE-HCP supports a low level file mapping that allows for quick compilationsof sets of files for a given subejct and data context.This is done in :func:`hcp.io.file_mapping.get_file_paths`, think of it as afile name synthesizer that takes certain data description parameters as inputsand lists all corresponding files.Example usage:.. code-block:: python>>> import hcp>>> files = hcp.file_mapping.et_file_paths(>>>     subject='123455', data_type='task_motor', output='raw',>>>     hcp_path='/media/storage/HCP')['/media/storage/HCP/123455/unprocessed/MEG/10-Motor/4D/c,rfDC','/media/storage/HCP/123455/unprocessed/MEG/10-Motor/4D/config']Why we are not globbing files? Because the HCP-MEG data are fixed, all filepatterns are known and access via Amazon web services easier if the filesto be accessed are known in advance.Gotchas=======Native coordinates and resulting plotting and processing peculartities----------------------------------------------------------------------The HCP for MEG provides coregistration information for native BTI/4Dsetting. MNE-Python expects coordinates in meters and the Neuromagright anterior superior (RAS) coordinates. However, essential information ismissing to compute all transforms needed to easily perform the conversions.For now, the way things work, all processing is performed in native BTI/4Dcoordinates with the device-to-head transform skipped (set to identity matrix),such that the coregistration directly maps from the native 4D sensors,represented in head coordinates, to the freesurfer space. This has a few minorconsequences that may be confusing to MNE-Python users.1. In the reader code you will see many flags set to ```convert=False```, etc.This is not a bug.2. All channel names and positions are native. Topographic plotting might notwork as as expected. First of all, the layout file is not recognized. Second,the coordinates are not regonized as native ones, eventually rotating anddistorting the graphical display. To fix this, either a proper layout can becomputed with :func:`hcp.viz.make_hcp_bti_layout`.Or the conversion to MNE can also beperformed using :func:`hcp.preprocessing.map_ch_coords_to_mne`.But note that source localization will be wrong when computed on data inNeuromag coordinates. As things are, coordinates have to be kept in the nativespace to be aligned with the HCP outputs.Reproducing HCP sensor space outputs------------------------------------A couple of steps are necessary to reproduce the original sensor space outputs.1. Reference channels should be regressed out. Checkout :func:`hcp.preprocessing.apply_ref_correction`.2. The trial info structure gives the correct latencies of the eventsThe latencies in the trigger channel are shifted by around 18 ms.For now we'd recommend using the events from the function :func:`hcp.read_trial_info`.3. The default filters in MNE and FieldTrip are different.FieldTrip uses a 4th order butterworth filter. In MNE you might needto adjust the `*_trans_bandwidth` parameter to avoid numerical errors.In the HCP outputs, evoked responses were filtered between 0.5 and 30Hz priorto baseline correction.4. Annotations need to be loaded and registered. The HCP consortium ships annotations of bad segments and bad channels.These have to be read and used. Check out `hcp.read_annot` and add badchannel names to `raw.info['bads']` and create and set an `mne.Annotations`object as attribute to `raw`, see below... code-block:: pythonannots = hcp.read_annot(subject, data_type, hcp_path=hcp_path,run_index=run_index)bad_segments = annots['segments']['all'] / raw.info['sfreq']raw.annotations = mne.Annotations(bad_segments[:, 0], (bad_segments[:, 1] - bad_segments[:, 0]),description='bad')5. ICA components related to eye blinks and heart beats need to be removedfrom the data. Checkout the ICA slot in the output of`hcp.read_annot` to get the HCP ICA components.Convenience functions---------------------NNE-HCP includes convenience functions that help setting up directory and file layoutsexpected by MNE-Python.:func:`hcp.make_mne_anatomy` will produce an MNE and Freesurfer compatible directory layout and will create the following outputs by default, mostly using sympbolic links:.. code-block:: bash$subjects_dir/$subject/bem/inner_skull.surf$subjects_dir/$subject/label/*$subjects_dir/$subject/mri/*$subjects_dir/$subject/surf/*$recordings_path/$subject/$subject-head_mri-trans.fifThese can then be set as $SUBJECTS_DIR and as MEG directory, consistentwith MNE examples.Here, `inner_skull.surf` and `$subject-head_mri-trans.fif` are written by the function such that they can be used by MNE. The latter is the coregistration matrix.Python Indexing^^^^^^^^^^^^^^^MNE-HCP corrects on reading the indices it finds for data segments, events, orcomponents. The indices it reads from the files will already be mapped toPython conventions by subtracting 1.Contributions-------------- currently `@dengemann` is pushing frequently to master, if you plan to contribute, open issues and pull requests, or contact `@dengemann` directly. Discussions are welcomed.Acknowledgements================This project is supported by the Amazon Webservices Research grant issued to Denis A. Engemann andby the ERC starting grant ERC StG 263584 issued to Virginie van Wassenhove.I acknowledge support by Alex Gramfort, Mainak Jas, Jona Sassenhagen, Giorgos Michalareas,Eric Larson, Danilo Bzdok, and Jan-Mathijs Schoffelen for discussions,inputs and help with finding the best way to mapHCP data to the MNE world.