I am working on the file "0. Metadata/0.0. Visualizing Metadata.ipynb".
The first task is:
0.0. Visualizing Metadata
RDF (Resource Description Framework) is a data model for information on the internet. It can be used to describe just about anything, but is usually applied to bibliographic collections: representing metadata about published documents.
RDF XML is a grammar/serialization format for representing RDF. There are other ways of representing RDF, like N-triples, Turtle, and JSON-LD.
One of the core concepts of RDF is representing (meta)data as a graph. Every element of the RDF document (a file containing RDF statements) is a node in the graph: articles, people, journals, literals (like volume numbers), etc. These nodes are called resources. Resources are linked together in triples: tri-partite statements consisting of a subject, a predicate, and an object.
In this notebook, we'll convert a simple Zotero RDF/XML document into a GraphML graph (a graph serialization format) and visualize that graph using Cytoscape.
--------   --------------------------------------------
Note   This exercise is intended only to introduce you to RDF and graphs, and isn't something that you are likely to do as part of an analysis. There is a sample RDF/XML file included in the data subdirectory, describing a single document. Use that file to start. You can try this with your own RDF if you want, but even a moderate number of documents will lead to extremely large and unweildy graphs. So, be careful.
--------   --------------------------------------------
Can you write Python code for it?

import rdflib
import networkx as nx
import os

rdf_path = 'data/example.rdf'


I am working on the file "0. Metadata/0.0. Visualizing Metadata.ipynb".
The first task is:
The code-cell below will create a new GraphML file that we can import in Cytoscape.
Can you write Python code for it?

graphml_path = 'output/example.graphml'
nx.write_graphml(graph, graphml_path)


I am working on the file "0. Preamble.ipynb".
The first task is:
2. Check installeded Versions
Can you write Python code for it?

import numpy
print('numpy:', numpy.__version__)

import scipy
print('scipy:', scipy.__version__)

import matplotlib
print('matplotlib:', matplotlib.__version__)

import IPython
print('iPython:', IPython.__version__)

import sklearn
print('scikit-learn:', sklearn.__version__)

import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)


I am working on the file "0. Python (Basic) Explore U.S. Births/Basics.ipynb".
The first task is:
3: Calculating Number Of Births Each Month
Now that the data is in a more usable format, we can start to analyze it. Let's calculate the total number of births that occured in each month, across all of the years in the dataset. We'll create a dictionary where each key is a unique month and each value is the number of births that happened in that month, across all years:
Can you write Python code for it?

def month_births(input_ls):
    births_per_month = {}
    for each in input_ls:
        month = each[1]
        births = each[4]
        if month in births_per_month:
            births_per_month[month] = births_per_month[month] + births
        else: births_per_month[month] = births
    return births_per_month

cdc_month_births = month_births(cdc_list)
cdc_month_births


I am working on the file "0. Python (Basic) Explore U.S. Births/Basics.ipynb".
The first task is:
Calculating Number Of Births Each Day Of Week
Can you write Python code for it?

def dow_births(input_ls):
    b_per_day = {}
    for each in input_ls:
        day_of_week = each[3]
        births = each[4]
        if day_of_week in b_per_day:
            b_per_day[day_of_week] = b_per_day[day_of_week] + births
        else:
            b_per_day[day_of_week] = births
    return b_per_day

cdc_day_births = dow_births(cdc_list)
cdc_day_births


I am working on the file "0. Python (Basic) Explore U.S. Births/Basics.ipynb".
The first task is:
Converting Data Into A List Of Lists
to convert the dataset into a list of lists where each nested list contains integer values (not strings). We also need to remove the header row.
Can you write Python code for it?

def read_csv(input_csv):
    f = open(input_csv, 'r')
    data = f.read()
    splited = data.split('\n')
    string_list = splited[1:len(splited)]
    final_list = []

    for each in string_list:
        int_fields = []
        string_fields = each.split(',')
        for each in string_fields:
            int_fields.append(int(each))
        final_list.append(int_fields)
    return final_list

cdc_list = read_csv("US_births_1994-2003_CDC_NCHS.csv")

cdc_list[0:10]


I am working on the file "0. Python (Basic) Explore U.S. Births/Basics.ipynb".
The first task is:
Creating A More General Function
it's better to create a single function that works for any column and specify the column we want as a parameter each time we call the function.
Can you write Python code for it?

def calc_counts(input_ls, column):
    dictionary = {}
    for each in input_ls:
        births = each[4]
        key = each[column]
        if key in dictionary:
            dictionary[key] = dictionary[key] + births
        else:
            dictionary[key] = births
    return dictionary

cdc_year_births = calc_counts(cdc_list, 0)
cdc_year_births

cdc_month_births = calc_counts(cdc_list, 1)
cdc_month_births

cdc_dom_births = calc_counts(cdc_list, 2)
cdc_dom_births

cdc_dow_births = calc_counts(cdc_list, 3)
cdc_dow_births

def calc_min(dict_ls):
    values = dict_ls.values
    min_value = 1
    max_value = 0
    for each in values:
        print(each)
        if each < (each + 1):
            min_value = each
    for each in values:
        if each > (each+1):
            max_value = each
    return max_value, min_value

g = calc_min(cdc_dow_births)
g

min_value = cdc_dow_births.values()
min_value


I am working on the file "0. Практика по NumPy.ipynb".
The first task is:
Задача 1: Переверните список и оставьте только слова длиннее трех букв.
Can you write Python code for it?

a = ['abc', 'hdok', 'banc', 'xkm', 'pckdax']

#b = a[::-1]
b= []
for i in a:
    if (len(i) > 3):
        b.append(i)

b = b[::-1]
b


I am working on the file "0. Практика по NumPy.ipynb".
The first task is:
Задача 2: Сделайте из списка списков одноуровневый список, в котором будут идти подряд все элементы
Can you write Python code for it?


I am working on the file "0. Практика по NumPy.ipynb".
The first task is:
Задача 4: Вычтите из данного массива единичную матрицу и транспонируйте результат
Can you write Python code for it?


I am working on the file "0. Практика по NumPy.ipynb".
The first task is:
Задача 5: Создайте функцию, которая находит то же значение, что и $F$, только принимает как параметр лишь размер матрицы $N$.
Can you write Python code for it?

N = 10

a = numpy.array([range(1, N**2 + 1)]).reshape(N, -1)
a

F = lambda matrix: numpy.sum(numpy.diag(matrix))

F(a)

MyF = lambda K:  numpy.sum(numpy.diag(numpy.array([range(1, K**2 + 1)]).reshape(K, -1)))

MyF(N)


I am working on the file "0.0_Introduction.ipynb".
The first task is:
<code>**</code> operator can be used to calculate powers
Can you write Python code for it?


I am working on the file "0.0_Introduction.ipynb".
The first task is:
Access a character in string using index, first index is 0
Indices may also be negative numbers, to start counting from the right
Can you write Python code for it?

word = 'python'
print word[0]
print word[-1]
print word[-2]


I am working on the file "0.0_Introduction.ipynb".
The first task is:
Concatenate lists using + operator
Add new items at the end of the list by using append() method
Calculate length of list using len() function
Can you write Python code for it?

print squares + [36, 49]

squares.append(36)
print squares

print len(squares)


I am working on the file "0.0_Introduction.ipynb".
The first task is:
However, unlike strings, list are mutable type. It is possible to change their content
Can you write Python code for it?

cubes = [1, 8, 27, 65, 125] # something's wrong here
print cubes

cubes[3] = 64
print cubes


I am working on the file "0.0_Introduction.ipynb".
The first task is:
Number of variables and length of sequence must match
Can you write Python code for it?


I am working on the file "0.0_Introduction.ipynb".
The first task is:
Print a string using print statement.
If you don’t want characters prefaced by \ to be interpreted as special characters, you can use raw strings by adding an r before the first quote
Can you write Python code for it?

print 'I can eat glass.\nIt doesn\'t hurt me'
print '---------------'
print r'I can eat glass.\n It doesn\'t hurt me'


I am working on the file "0.0_Introduction.ipynb".
The first task is:
Python strings cannot be changed - they are immutable
Can you write Python code for it?

# get error when trying to change
word[0] = 'J'


I am working on the file "0.0_Introduction.ipynb".
The first task is:
What do we see?

No semicolon
No parenthese () or brackets {}
Wait... How do we express code block like in C?
Using indentation

No static typing

Basic of Python
Number
Expression syntax and the operators: <code>+, -, *, %, /</code> work just like in most other languages.
Can you write Python code for it?


I am working on the file "0.1/binary_classification.ipynb".
The first task is:
Also, the feature importances can be obtained if the algorithm has the option.
Can you write Python code for it?

importances = mc.feature_importances('RFC')

%matplotlib inline
import matplotlib.pyplot as plt

indices = range(n_features)
f, ax = plt.subplots(figsize=(11, 9))
plt.title("Feature importances", fontsize = 20)
plt.bar(indices, importances, color="black", align="center")

plt.xticks(indices)
plt.ylabel("Importance", fontsize = 18)
plt.xlabel("Index of the feature", fontsize = 18)


I am working on the file "0.1/binary_classification.ipynb".
The first task is:
In order to analize an especific fold, you can obtaine the indices of the data used for training and testing, the model trained, as well as the prediction on the test data. The method best_estimator() has the parameter fold_key, if it is not set, the method returns the fold with the highest accuracy.
TODO: Use the measurement as a parameter to get the best estimator
Can you write Python code for it?

# Get the results of the parition that has the high accuracy

fold, bm_model, bm_y_pred, bm_train_indices, bm_test_indices = mc.best_estimator('RFC')['RFC']

print(">>Best model in fold: {}".format(fold))
print(">>>Trained model \n{}".format(bm_model))
print(">>>Predicted labels: \n{}".format(bm_y_pred))
print(">>>Indices of the samples used for training: \n{}".format(bm_train_indices))
print(">>>Indices of samples used for predicting: \n{}".format(bm_test_indices))


I am working on the file "0.1/binary_classification.ipynb".
The first task is:
In the case that you need to train the model again using the data of an especific fold, you can use the bm_train_indices and bm_test_indices.
Can you write Python code for it?

# Recover the partition of the dataset based on the results of the best model
X_train_final, X_test_final = X[bm_train_indices], X[bm_test_indices]
y_train_final, y_test_final = y[bm_train_indices], y[bm_test_indices]

# Testing the best model using again all the training set
bm_model.fit(X_train_final, y_train_final)
print("Final score {0:.4f}".format(bm_model.score(X_test_final, y_test_final)))


I am working on the file "0.1/binary_classification.ipynb".
The first task is:
In this example, the breast cancer wisconsin dataset contains 569 samples and 30 features, details of the dataset can be find here.
Can you write Python code for it?

# Load data
cancer = datasets.load_breast_cancer()
X = cancer.data
y = cancer.target
n_samples, n_features = X.shape
print("samples:{}, features:{}".format(n_samples, n_features))


I am working on the file "0.1/binary_classification.ipynb".
The first task is:
The MultiClassifier trains the multiple estimators previouly configured. First, the data is divided n_splits times, in this case in 5 folds using the StratifiedKFold class. As it is shown in the following table, the data is divided 5 times, four of the 5 blocks will be used for training (blue ones), while one will be used for testing (orange one). In addition, if the parameter shuffle=True, the data will be rearranged before splitting into blocks.
<img src="./images/folds.png" alt="" height="250" width="250">
Can you write Python code for it?

# Training
mc = MultiClassifier(n_splits=5, shuffle=True, random_state=random_state)


I am working on the file "0.1/binary_classification.ipynb".
The first task is:
This section is used to configure the classifiers that are going to be used in the classification, in this case, the Support Vector Machines (SVC) and Random Forest classifiers (RF) are going to be applied. The dictionary models is used to declare the classifiers and their parameters that are NOT going to be tunned in the cross-validation. The dictionary model_params is used to specify the parameters that DO will be tunned in the cross-validation. The dictionary  cv_params is used to configure how the grid cross-validation is going to be performed.
Can you write Python code for it?

# Configuration

random_state = 2017 # seed used by the random number generator

models = {
    # NOTE: SVC and RFC are the names that will be used to make reference to the models after the training step.
    'SVC': SVC(probability=True,
               random_state=random_state),
    'RFC': RandomForestClassifier(random_state=random_state)
}

model_params = {
    'SVC': {'kernel':['linear', 'rbf', 'sigmoid']},
    'RFC': {'n_estimators': [25,50, 75, 100]}
}

cv_params = {
    'cv': StratifiedKFold(n_splits=3, shuffle=False, random_state=random_state)
}


I am working on the file "0.1/multiclass_classification.ipynb".
The first task is:
In order to analize an especific fold, you can obtaine the indices of the data used for training and testing, the model trained, as well as the prediction on the test data. The method best_estimator() has the parameter fold_key, if it is not set, the method returns the fold with the highest accuracy.
TODO: Use the measurement as a parameter to get the best estimator
Can you write Python code for it?


I am working on the file "0.1/multiclass_classification.ipynb".
The first task is:
In the case that you need to train the model again using the data of an especific fold, you can use the bm_train_indices and bm_test_indices.
Can you write Python code for it?


I am working on the file "0.1/multiclass_classification.ipynb".
The first task is:
In this example, the wine dataset contains 178 samples, 13 features, and 3 clases. Details of the dataset can be find here.
Can you write Python code for it?

# Load data
iris = load_iris()
X = iris.data
y = iris.target
n_samples, n_features = X.shape
print("samples:{}, features:{}, labels:{}".format(n_samples, n_features, np.unique(y)))

# Preprocessing
std_scale = preprocessing.StandardScaler().fit(X)
X_std = std_scale.transform(X)
print('After standardization:{:.4f}, {:.4f}'.format(X_std.mean(), X_std.std()))


I am working on the file "0.1/multiclass_classification.ipynb".
The first task is:
Third, once that the best parameters were obtained, a model is generated from the training data. Following the example, this model is then tested on the fold_1:test block. As soon as the training and test were perfromed for each fold, the results can be visualized in a report.
Can you write Python code for it?


I am working on the file "0.12.14 bugs - test in 0.12.5.ipynb".
The first task is:
Responsive in notebook
Spills a scroll bar.
Not a problem in a vanilla save file.
Bad in 0.12.6, 0.12.7, 0.12.9, 0.12.10, 0.12.14, 0.12.15dev3
God in 0.12.5
Can you write Python code for it?

!conda list | egrep "jupyter|notebook"

p = figure(plot_height=200, sizing_mode='scale_width')
p.scatter(x, y, alpha=0.1)
show(p)

N = 4000
x = np.random.random(size=N) * 100
y = np.random.random(size=N) * 100
radii = np.random.random(size=N) * 1.5
colors = [
    "#%02x%02x%02x" % (int(r), int(g), 150) for r, g in zip(50+2*x, 30+2*y)
]

TOOLS="hover,crosshair,pan,wheel_zoom,box_zoom,undo,redo,reset,tap,save,box_select,poly_select,lasso_select,"

p = figure(tools=TOOLS, sizing_mode='scale_width')

p.scatter(x, y, radius=radii,
          fill_color=colors, fill_alpha=0.6,
          line_color=None)

show(p)

save(p, 'color_scatter.html')


I am working on the file "0.12.14 bugs - test in 0.12.5.ipynb".
The first task is:
WebGL problems
Drag around canvas is shifted down, cut off at top spilling over bottom.
Bad in 0.12.14 and 0.12.15dev3
Good in 0.12.10
Can you write Python code for it?

N = 10000

x = np.random.normal(0, np.pi, N)
y = np.sin(x) + np.random.normal(0, 0.2, N)

p = figure(webgl=True)
p.scatter(x, y, alpha=0.1)
show(p)


I am working on the file "0.12.14 bugs - test in latest dev.ipynb".
The first task is:
WebGL problems
Drag around canvas is shifted down, cut off at top spilling over bottom
Can you write Python code for it?

N = 10000

x = np.random.normal(0, np.pi, N)
y = np.sin(x) + np.random.normal(0, 0.2, N)

p = figure(output_backend="webgl")
p.scatter(x, y, alpha=0.1)
show(p)


I am working on the file "0.12.14 bugs.ipynb".
The first task is:
Responsive in notebook
Spills a scroll bar.
Not a problem in a vanilla save file.
Can you write Python code for it?

!conda list | egrep "jupyter|notebook"

p = figure(plot_height=200, sizing_mode='scale_width')
p.scatter(x, y, alpha=0.1)
show(p)

N = 4000
x = np.random.random(size=N) * 100
y = np.random.random(size=N) * 100
radii = np.random.random(size=N) * 1.5
colors = [
    "#%02x%02x%02x" % (int(r), int(g), 150) for r, g in zip(50+2*x, 30+2*y)
]

TOOLS="hover,crosshair,pan,wheel_zoom,zoom_in,zoom_out,box_zoom,undo,redo,reset,tap,save,box_select,poly_select,lasso_select,"

p = figure(tools=TOOLS, sizing_mode='scale_width')

p.scatter(x, y, radius=radii,
          fill_color=colors, fill_alpha=0.6,
          line_color=None)

show(p)

save(p, 'color_scatter.html')


I am working on the file "0.12/_downloads/plot_artifacts_correction_filtering.ipynb".
The first task is:
.. _tut_artifacts_filter:
Filtering and Resampling
Certain artifacts are restricted to certain frequencies and can therefore
be fixed by filtering. An artifact that typically affects only some
frequencies is due to the power line.
Power-line noise is a noise created by the electrical network.
It is composed of sharp peaks at 50Hz (or 60Hz depending on your
geographical location). Some peaks may also be present at the harmonic
frequencies, i.e. the integer multiples of
the power-line frequency, e.g. 100Hz, 150Hz, ... (or 120Hz, 180Hz, ...).
Can you write Python code for it?

import numpy as np
import mne
from mne.datasets import sample

data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_raw.fif'
proj_fname = data_path + '/MEG/sample/sample_audvis_eog_proj.fif'

tmin, tmax = 0, 20  # use the first 20s of data

# Setup for reading the raw data (save memory by cropping the raw data
# before loading it)
raw = mne.io.read_raw_fif(raw_fname).crop(tmin, tmax).load_data()
raw.info['bads'] = ['MEG 2443', 'EEG 053']  # bads + 2 more

fmin, fmax = 2, 300  # look at frequencies between 2 and 300Hz
n_fft = 2048  # the FFT size (n_fft). Ideally a power of 2

# Pick a subset of channels (here for speed reason)
selection = mne.read_selection('Left-temporal')
picks = mne.pick_types(raw.info, meg='mag', eeg=False, eog=False,
                       stim=False, exclude='bads', selection=selection)

# Let's first check out all channel types
raw.plot_psd(area_mode='range', tmax=10.0, picks=picks)


I am working on the file "0.12/_downloads/plot_artifacts_correction_filtering.ipynb".
The first task is:
Down-sampling (for performance reasons)
When performing experiments where timing is critical, a signal with a high
sampling rate is desired. However, having a signal with a much higher
sampling rate than necessary needlessly consumes memory and slows down
computations operating on the data. To avoid that, you can down-sample
your time series.
Data resampling can be done with resample methods.
Can you write Python code for it?

raw.resample(100, npad="auto")  # set sampling frequency to 100Hz
raw.plot_psd(area_mode='range', tmax=10.0, picks=picks)


I am working on the file "0.12/_downloads/plot_artifacts_correction_filtering.ipynb".
The first task is:
High-pass filtering to remove slow drifts
If you're only interested in low frequencies, below the peaks of power-line
noise you can simply high pass filter the data.
Can you write Python code for it?

raw.filter(1., None)  # low pass filtering above 1 Hz
raw.plot_psd(area_mode='range', tmax=10.0, picks=picks)


I am working on the file "0.12/_downloads/plot_artifacts_correction_filtering.ipynb".
The first task is:
To do the low-pass and high-pass filtering in one step you can do
a so-called band-pass filter by running
Can you write Python code for it?

raw.filter(1., 50.)  # band-pass filtering in the range 1 Hz - 50 Hz


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
.. _tut_artifacts_correct_ica:
Artifact Correction with ICA
ICA finds directions in the feature space
corresponding to projections with high non-Gaussianity. We thus obtain
a decomposition into independent components, and the artifact's contribution
is localized in only a small number of components.
These components have to be correctly identified and removed.
If EOG or ECG recordings are available, they can be used in ICA to
automatically select the corresponding artifact components from the
decomposition. To do so, you have to first build an Epoch object around
blink or heartbeat event.
Can you write Python code for it?

import numpy as np

import mne
from mne.datasets import sample

from mne.preprocessing import ICA
from mne.preprocessing import create_eog_epochs

# getting some data ready
data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'

raw = mne.io.read_raw_fif(raw_fname, preload=True)
raw.filter(1, 40, n_jobs=2)  # 1Hz high pass is often helpful for fitting ICA

picks_meg = mne.pick_types(raw.info, meg=True, eeg=False, eog=False,
                           stim=False, exclude='bads')


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Advanced artifact detection
Let's use a more efficient way to find artefacts
Can you write Python code for it?

eog_average = create_eog_epochs(raw, reject=dict(mag=5e-12, grad=4000e-13),
                                picks=picks_meg).average()

# We simplify things by setting the maximum number of components to reject
n_max_eog = 1  # here we bet on finding the vertical EOG components
eog_epochs = create_eog_epochs(raw, reject=reject)  # get single EOG trials
eog_inds, scores = ica.find_bads_eog(eog_epochs)  # find via correlation

ica.plot_scores(scores, exclude=eog_inds)  # look at r scores of components
# we can see that only one component is highly correlated and that this
# component got detected by our correlation analysis (red).

ica.plot_sources(eog_average, exclude=eog_inds)  # look at source time course


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Before applying artifact correction please learn about your actual artifacts
by reading :ref:tut_artifacts_detect.
Fit ICA
ICA parameters:
Can you write Python code for it?

n_components = 25  # if float, select n_components by explained variance of PCA
method = 'fastica'  # for comparison with EEGLAB try "extended-infomax" here
decim = 3  # we need sufficient statistics, not all time points -> save time


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Do not copy this at home! You start by reading in a collections of ICA
solutions, something like
icas = [mne.preprocessing.read_ica(fname) for fname in ica_fnames]
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Exercise: find and remove ECG artifacts using ICA!
What if we don't have an EOG channel?
1) make a bipolar reference from frontal EEG sensors and use as virtual EOG
channel. This can be tricky though as you can only hope that the frontal
EEG channels only reflect EOG and not brain dynamics in the prefrontal
cortex.
2) Go for a semi-automated approach, using template matching.
In MNE-Python option 2 is easily achievable and it might be better,
so let's have a look at it.
Can you write Python code for it?

from mne.preprocessing.ica import corrmap  # noqa


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Indeed it looks like an EOG, also in the average time course.
So our template shall be a tuple like (reference_run_index, component_index):
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Now we can do the corrmap.
Can you write Python code for it?

fig_template, fig_detected = corrmap(
    icas_from_other_data, template=template, label="blinks", show=True,
    threshold=.8, ch_type='mag')


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
The idea behind corrmap is that artefact patterns are similar across subjects
and can thus be identified by correlating the different patterns resulting
from each solution with a template. The procedure is therefore
semi-automatic. Corrmap hence takes at least a list of ICA solutions and a
template, that can be an index or an array. As we don't have different
subjects or runs available today, here we will fit ICA models to different
parts of the recording and then use as a user-defined template the ICA
that we just fitted for detecting corresponding components in the three "new"
ICAs. The following block of code addresses this point and should not be
copied, ok?
We'll start by simulating a group of subjects or runs from a subject
Can you write Python code for it?

start, stop = [0, len(raw.times) - 1]
intervals = np.linspace(start, stop, 4, dtype=int)
icas_from_other_data = list()
raw.pick_types(meg=True, eeg=False)  # take only MEG channels
for ii, start in enumerate(intervals):
    if ii + 1 < len(intervals):
        stop = intervals[ii + 1]
        print('fitting ICA from {0} to {1} seconds'.format(start, stop))
        this_ica = ICA(n_components=n_components, method=method).fit(
            raw, start=start, stop=stop, reject=reject)
        icas_from_other_data.append(this_ica)


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
These labels were used by the plotters and are added automatically
by artifact detection functions. You can also manually edit them to annotate
components.
Now let's see how we would modify our signals if we would remove this
component from the data
Can you write Python code for it?

ica.plot_overlay(eog_average, exclude=eog_inds, show=False)
# red -> before, black -> after. Yes! We remove quite a lot!

# to definitely register this component as a bad one to be removed
# there is the ``ica.exclude`` attribute, a simple Python list

ica.exclude.extend(eog_inds)

# from now on the ICA will reject this component even if no exclude
# parameter is passed, and this information will be stored to disk
# on saving

# uncomment this for reading and writing
# ica.save('my-ica.fif')
# ica = read_ica('my-ica.fif')


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Which one is the bad EOG component?
Here we rely on our previous detection algorithm. You will need to decide
yourself in that situation where no other detection is available.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
use our previous ICA as reference.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
we avoid fitting ICA on crazy environmental artifacts that would
dominate the variance and decomposition
Can you write Python code for it?

reject = dict(mag=5e-12, grad=4000e-13)
ica.fit(raw, picks=picks_meg, decim=decim, reject=reject)
print(ica)


I am working on the file "0.12/_downloads/plot_artifacts_correction_maxwell_filtering.ipynb".
The first task is:
Artifact correction with Maxwell filter
This tutorial shows how to clean MEG data with Maxwell filtering.
Maxwell filtering in MNE can be used to suppress sources of external
intereference and compensate for subject head movements.
See :ref:maxwell for more details.
Can you write Python code for it?

import mne
from mne.preprocessing import maxwell_filter

data_path = mne.datasets.sample.data_path()


I am working on the file "0.12/_downloads/plot_artifacts_correction_maxwell_filtering.ipynb".
The first task is:
Preprocess with Maxwell filtering
Can you write Python code for it?

raw = mne.io.read_raw_fif(raw_fname)
raw.info['bads'] = ['MEG 2443', 'EEG 053', 'MEG 1032', 'MEG 2313']  # set bads
# Here we don't use tSSS (set st_duration) because MGH data is very clean
raw_sss = maxwell_filter(raw, cross_talk=ctc_fname, calibration=fine_cal_fname)


I am working on the file "0.12/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
.. _tut_artifacts_reject:
Rejecting bad data (channels and segments)
Can you write Python code for it?

import numpy as np
import mne
from mne.datasets import sample

data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
raw = mne.io.read_raw_fif(raw_fname)


I am working on the file "0.12/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
And plot the so-called drop log that details the reason for which some
epochs have been dropped.
Can you write Python code for it?

print(epochs.drop_log[40:45])  # only a subset
epochs.plot_drop_log()


I am working on the file "0.12/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
As the data is epoched, all the epochs overlapping with segments whose
description starts with 'bad' are rejected by default. To turn rejection off,
use keyword argument reject_by_annotation=False when constructing
:class:mne.Epochs. When working with neuromag data, the first_samp
offset of raw acquisition is also taken into account the same way as with
event lists. For more see :class:mne.Epochs and :class:mne.Annotations.
.. _rejecting_bad_epochs:
Rejecting bad epochs
When working with segmented data (Epochs) MNE offers a quite simple approach
to automatically reject/ignore bad epochs. This is done by defining
thresholds for peak-to-peak amplitude and flat signal detection.
In the following code we build Epochs from Raw object. One of the provided
parameter is named reject. It is a dictionary where every key is a
channel type as a sring and the corresponding values are peak-to-peak
rejection parameters (amplitude ranges as floats). Below we define
the peak-to-peak rejection values for gradiometers,
magnetometers and EOG:
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
Let's now interpolate the bad channels (displayed in red above)
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
Let's plot the cleaned data
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
We then drop/reject the bad epochs
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
Why setting a channel bad?: If a channel does not show
a signal at all (flat) it is important to exclude it from the
analysis. If a channel as a noise level significantly higher than the
other channels it should be marked as bad. Presence of bad channels
can have terribe consequences on down stream analysis. For a flat channel
some noise estimate will be unrealistically low and
thus the current estimate calculations will give a strong weight
to the zero signal on the flat channels and will essentially vanish.
Noisy channels can also affect others when signal-space projections
or EEG average electrode reference is employed. Noisy bad channels can
also adversely affect averaging and noise-covariance matrix estimation by
causing unnecessary rejections of epochs.
Recommended ways to identify bad channels are:

Observe the quality of data during data
  acquisition and make notes of observed malfunctioning channels to
  your measurement protocol sheet.

View the on-line averages and check the condition of the channels.

Compute preliminary off-line averages with artifact rejection,
  SSP/ICA, and EEG average electrode reference computation
  off and check the condition of the channels.

View raw data with :func:mne.io.Raw.plot without SSP/ICA
  enabled and identify bad channels.

.. note::
    Setting the bad channels should be done as early as possible in the
    analysis pipeline. That's why it's recommended to set bad channels
    the raw objects/files. If present in the raw data
    files, the bad channel selections will be automatically transferred
    to averaged files, noise-covariance matrices, forward solution
    files, and inverse operator decompositions.
The actual removal happens using :func:pick_types <mne.pick_types> with
exclude='bads' option (see :ref:picking_channels).
Instead of removing the bad channels, you can also try to repair them.
This is done by interpolation of the data from other channels.
To illustrate how to use channel interpolation let us load some data.
Can you write Python code for it?

# Reading data with a bad channel marked as bad:
fname = data_path + '/MEG/sample/sample_audvis-ave.fif'
evoked = mne.read_evokeds(fname, condition='Left Auditory',
                          baseline=(None, 0))

# restrict the evoked to EEG and MEG channels
evoked.pick_types(meg=True, eeg=True, exclude=[])

# plot with bads
evoked.plot(exclude=[])

print(evoked.info['bads'])


I am working on the file "0.12/_downloads/plot_artifacts_correction_ssp.ipynb".
The first task is:
.. _tut_artifacts_correct_ssp:
Artifact Correction with SSP
Can you write Python code for it?

import numpy as np

import mne
from mne.datasets import sample
from mne.preprocessing import compute_proj_ecg, compute_proj_eog

# getting some data ready
data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'

raw = mne.io.read_raw_fif(raw_fname, preload=True)
raw.pick_types(meg=True, ecg=True, eog=True, stim=True)


I am working on the file "0.12/_downloads/plot_artifacts_correction_ssp.ipynb".
The first task is:
Apply SSP projections
MNE is handling projections at the level of the info,
so to register them populate the list that you find in the 'proj' field
Can you write Python code for it?

raw.info['projs'] += eog_projs + ecg_projs


I am working on the file "0.12/_downloads/plot_artifacts_correction_ssp.ipynb".
The first task is:
Compute SSP projections
Can you write Python code for it?

projs, events = compute_proj_ecg(raw, n_grad=1, n_mag=1, average=True)
print(projs)

ecg_projs = projs[-2:]
mne.viz.plot_projs_topomap(ecg_projs)

# Now for EOG

projs, events = compute_proj_eog(raw, n_grad=1, n_mag=1, average=True)
print(projs)

eog_projs = projs[-2:]
mne.viz.plot_projs_topomap(eog_projs)


I am working on the file "0.12/_downloads/plot_artifacts_correction_ssp.ipynb".
The first task is:
Yes this was it. Now MNE will apply the projs on demand at any later stage,
so watch out for proj parmeters in functions or to it explicitly
with the .apply_proj method
Demonstrate SSP cleaning on some evoked data
Can you write Python code for it?

events = mne.find_events(raw, stim_channel='STI 014')
reject = dict(grad=4000e-13, mag=4e-12, eog=150e-6)
# this can be highly data dependent
event_id = {'auditory/left': 1}

epochs_no_proj = mne.Epochs(raw, events, event_id, tmin=-0.2, tmax=0.5,
                            proj=False, baseline=(None, 0), reject=reject)
epochs_no_proj.average().plot(spatial_colors=True)

epochs_proj = mne.Epochs(raw, events, event_id, tmin=-0.2, tmax=0.5, proj=True,
                         baseline=(None, 0), reject=reject)
epochs_proj.average().plot(spatial_colors=True)


I am working on the file "0.12/_downloads/plot_artifacts_detection.ipynb".
The first task is:
.. _tut_artifacts_detect:
Introduction to artifacts and artifact detection
Since MNE supports the data of many different acquisition systems, the
particular artifacts in your data might behave very differently from the
artifacts you can observe in our tutorials and examples.
Therefore you should be aware of the different approaches and of
the variability of artifact rejection (automatic/manual) procedures described
onwards. At the end consider always to visually inspect your data
after artifact rejection or correction.
Background: what is an artifact?
Artifacts are signal interference that can be
endogenous (biological) and exogenous (environmental).
Typical biological artifacts are head movements, eye blinks
or eye movements, heart beats. The most common environmental
artifact is due to the power line, the so-called line noise.
How to handle artifacts?
MNE deals with artifacts by first identifying them, and subsequently removing
them. Detection of artifacts can be done visually, or using automatic routines
(or a combination of both). After you know what the artifacts are, you need
remove them. This can be done by:
- *ignoring* the piece of corrupted data
- *fixing* the corrupted data

For the artifact detection the functions MNE provides depend on whether
your data is continuous (Raw) or epoch-based (Epochs) and depending on
whether your data is stored on disk or already in memory.
Detecting the artifacts without reading the complete data into memory allows
you to work with datasets that are too large to fit in memory all at once.
Detecting the artifacts in continuous data allows you to apply filters
(e.g. a band-pass filter to zoom in on the muscle artifacts on the temporal
channels) without having to worry about edge effects due to the filter
(i.e. filter ringing). Having the data in memory after segmenting/epoching is
however a very efficient way of browsing through the data which helps
in visualizing. So to conclude, there is not a single most optimal manner
to detect the artifacts: it just depends on...
Can you write Python code for it?

import mne
from mne.datasets import sample
from mne.preprocessing import create_ecg_epochs, create_eog_epochs

# getting some data ready
data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_raw.fif'

raw = mne.io.read_raw_fif(raw_fname, preload=True)


I am working on the file "0.12/_downloads/plot_artifacts_detection.ipynb".
The first task is:
Low frequency drifts and line noise
Can you write Python code for it?

(raw.copy().pick_types(meg='mag')
           .del_proj(0)
           .plot(duration=60, n_channels=100, remove_dc=False))


I am working on the file "0.12/_downloads/plot_artifacts_detection.ipynb".
The first task is:
On MEG sensors we see narrow frequency peaks at 60, 120, 180, 240 Hz,
related to line noise.
But also some high amplitude signals between 25 and 32 Hz, hinting at other
biological artifacts such as ECG. These can be most easily detected in the
time domain using MNE helper functions
See :ref:tut_artifacts_filter.
ECG
finds ECG events, creates epochs, averages and plots
Can you write Python code for it?

average_ecg = create_ecg_epochs(raw).average()
print('We found %i ECG events' % average_ecg.nave)
average_ecg.plot_joint()


I am working on the file "0.12/_downloads/plot_artifacts_detection.ipynb".
The first task is:
we can see typical time courses and non dipolar topographies
not the order of magnitude of the average artifact related signal and
compare this to what you observe for brain signals
EOG
Can you write Python code for it?

average_eog = create_eog_epochs(raw).average()
print('We found %i EOG events' % average_eog.nave)
average_eog.plot_joint()


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Data channel array consisted of 274 MEG axial gradiometers, 26 MEG reference
sensors and 2 EEG electrodes (Cz and Pz).
In addition:
  - 1 stim channel for marking presentation times for the stimuli
  - 1 audio channel for the sent signal
  - 1 response channel for recording the button presses
  - 1 ECG bipolar
  - 2 EOG bipolar (vertical and horizontal)
  - 12 head tracking channels
  - 20 unused channels
The head tracking channels and the unused channels are marked as misc
channels. Here we define the EOG and ECG channels.
Can you write Python code for it?

raw.set_channel_types({'HEOG': 'eog', 'VEOG': 'eog', 'ECG': 'ecg'})
if not use_precomputed:
    # Leave out the two EEG channels for easier computation of forward.
    raw.pick_types(meg=True, eeg=False, stim=True, misc=True, eog=True,
                   ecg=True)


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Difference.
Can you write Python code for it?

stc_difference = apply_inverse(evoked_difference, inv, lambda2, 'dSPM')
brain = stc_difference.plot(subjects_dir=subjects_dir, subject=subject,
                            surface='inflated', time_viewer=False, hemi='lh')
brain.set_data_time_index(150)


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
For noise reduction, a set of bad segments have been identified and stored
in csv files. The bad segments are later used to reject epochs that overlap
with them.
The file for the second run also contains some saccades. The saccades are
removed by using SSP. We use pandas to read the data from the csv files. You
can also view the files with your favorite text editor.
Can you write Python code for it?

annotations_df = pd.DataFrame()
offset = n_times_run1
for idx in [1, 2]:
    csv_fname = op.join(data_path, 'MEG', 'bst_auditory',
                        'events_bad_0%s.csv' % idx)
    df = pd.read_csv(csv_fname, header=None,
                     names=['onset', 'duration', 'id', 'label'])
    print('Events from run {0}:'.format(idx))
    print(df)

    df['onset'] += offset * (idx - 1)
    annotations_df = pd.concat([annotations_df, df], axis=0)

saccades_events = df[df['label'] == 'saccade'].values[:, :3].astype(int)

# Conversion from samples to times:
onsets = annotations_df['onset'].values / raw.info['sfreq']
durations = annotations_df['duration'].values / raw.info['sfreq']
descriptions = map(str, annotations_df['label'].values)

annotations = mne.Annotations(onsets, durations, descriptions)
raw.annotations = annotations
del onsets, durations, descriptions


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Here we compute the saccade and EOG projectors for magnetometers and add
them to the raw data. The projectors are added to both runs.
Can you write Python code for it?

saccade_epochs = mne.Epochs(raw, saccades_events, 1, 0., 0.5, preload=True,
                            reject_by_annotation=False)

projs_saccade = mne.compute_proj_epochs(saccade_epochs, n_mag=1, n_eeg=0,
                                        desc_prefix='saccade')
if use_precomputed:
    proj_fname = op.join(data_path, 'MEG', 'bst_auditory',
                         'bst_auditory-eog-proj.fif')
    projs_eog = mne.read_proj(proj_fname)[0]
else:
    projs_eog, _ = mne.preprocessing.compute_proj_eog(raw.load_data(),
                                                      n_mag=1, n_eeg=0)
raw.add_proj(projs_saccade)
raw.add_proj(projs_eog)
del saccade_epochs, saccades_events, projs_eog, projs_saccade  # To save memory


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Here we plot the ERF of standard and deviant conditions. In both conditions
we can see the P50 and N100 responses. The mismatch negativity is visible
only in the deviant condition around 100-200 ms. P200 is also visible around
170 ms in both conditions but much stronger in the standard condition. P300
is visible in deviant condition only (decision making in preparation of the
button press). You can view the topographies from a certain time span by
painting an area with clicking and holding the left mouse button.
Can you write Python code for it?

evoked_std.plot(window_title='Standard', gfp=True)
evoked_dev.plot(window_title='Deviant', gfp=True)


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Source estimation.
We compute the noise covariance matrix from the empty room measurement
and use it for the other runs.
Can you write Python code for it?

reject = dict(mag=4e-12)
cov = mne.compute_raw_covariance(raw_erm, reject=reject)
cov.plot(raw_erm.info)
del raw_erm


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
The averages for each conditions are computed.
Can you write Python code for it?

evoked_std = epochs_standard.average()
evoked_dev = epochs_deviant.average()
del epochs_standard, epochs_deviant


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
The data was collected with a CTF 275 system at 2400 Hz and low-pass
filtered at 600 Hz. Here the data and empty room data files are read to
construct instances of :class:mne.io.Raw.
Can you write Python code for it?

data_path = bst_auditory.data_path()

subject = 'bst_auditory'
subjects_dir = op.join(data_path, 'subjects')

raw_fname1 = op.join(data_path, 'MEG', 'bst_auditory',
                     'S01_AEF_20131218_01.ds')
raw_fname2 = op.join(data_path, 'MEG', 'bst_auditory',
                     'S01_AEF_20131218_02.ds')
erm_fname = op.join(data_path, 'MEG', 'bst_auditory',
                    'S01_Noise_20131218_01.ds')


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
The epochs (trials) are created for MEG channels. First we find the picks
for MEG and EOG channels. Then the epochs are constructed using these picks.
The epochs overlapping with annotated bad segments are also rejected by
default. To turn off rejection by bad segments (as was done earlier with
saccades) you can use keyword reject_by_annotation=False.
Can you write Python code for it?

picks = mne.pick_types(raw.info, meg=True, eeg=False, stim=False, eog=True,
                       exclude='bads')

epochs = mne.Epochs(raw, events, event_id, tmin, tmax, picks=picks,
                    baseline=(None, 0), reject=reject, preload=False,
                    proj=True)


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
The event timing is adjusted by comparing the trigger times on detected
sound onsets on channel UADC001-4408.
Can you write Python code for it?

sound_data = raw[raw.ch_names.index('UADC001-4408')][0][0]
onsets = np.where(np.abs(sound_data) > 2. * np.std(sound_data))[0]
min_diff = int(0.5 * raw.info['sfreq'])
diffs = np.concatenate([[min_diff + 1], np.diff(onsets)])
onsets = onsets[diffs > min_diff]
assert len(onsets) == len(events)
diffs = 1000. * (events[:, 0] - onsets) / raw.info['sfreq']
print('Trigger delay removed (μ ± σ): %0.1f ± %0.1f ms'
      % (np.mean(diffs), np.std(diffs)))
events[:, 0] = onsets
del sound_data, diffs


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
The sources are computed using dSPM method and plotted on an inflated brain
surface. For interactive controls over the image, use keyword
time_viewer=True.
Standard condition.
Can you write Python code for it?

stc_standard = mne.minimum_norm.apply_inverse(evoked_std, inv, lambda2, 'dSPM')
brain = stc_standard.plot(subjects_dir=subjects_dir, subject=subject,
                          surface='inflated', time_viewer=False, hemi='lh')
brain.set_data_time_index(120)
del stc_standard, evoked_std, brain


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
To reduce memory consumption and running time, some of the steps are
precomputed. To run everything from scratch change this to False. With
use_precomputed = False running time of this script can be several
minutes even on a fast computer.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Typical preprocessing step is the removal of power line artifact (50 Hz or
60 Hz). Here we notch filter the data at 60, 120 and 180 to remove the
original 60 Hz artifact and the harmonics. Normally this would be done to
raw data (with :func:mne.io.Raw.filter), but to reduce memory consumption
of this tutorial, we do it at evoked stage.
Can you write Python code for it?

if use_precomputed:
    sfreq = evoked_std.info['sfreq']
    nchan = evoked_std.info['nchan']
    notches = [60, 120, 180]
    for ch_idx in range(nchan):
        evoked_std.data[ch_idx] = notch_filter(evoked_std.data[ch_idx], sfreq,
                                               notches, verbose='ERROR')
        evoked_dev.data[ch_idx] = notch_filter(evoked_dev.data[ch_idx], sfreq,
                                               notches, verbose='ERROR')
        evoked_std.data[ch_idx] = low_pass_filter(evoked_std.data[ch_idx],
                                                  sfreq, 100, verbose='ERROR')
        evoked_dev.data[ch_idx] = low_pass_filter(evoked_dev.data[ch_idx],
                                                  sfreq, 100, verbose='ERROR')


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Visually inspect the effects of projections. Click on 'proj' button at the
bottom right corner to toggle the projectors on/off. EOG events can be
plotted by adding the event list as a keyword argument. As the bad segments
and saccades were added as annotations to the raw data, they are plotted as
well.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
We also lowpass filter the data at 100 Hz to remove the hf components.
Can you write Python code for it?

if not use_precomputed:
    raw.filter(None, 100.)


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
We can see the MMN effect more clearly by looking at the difference between
the two conditions. P50 and N100 are no longer visible, but MMN/P200 and
P300 are emphasised.
Can you write Python code for it?

evoked_difference = combine_evoked([evoked_dev, evoked_std], weights=[1, -1])
evoked_difference.plot(window_title='Difference', gfp=True)


I am working on the file "0.12/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
We only use first 40 good epochs from each run. Since we first drop the bad
epochs, the indices of the epochs are no longer same as in the original
epochs collection. Investigation of the event timings reveals that first
epoch from the second run corresponds to index 182.
Can you write Python code for it?

epochs.drop_bad()
epochs_standard = mne.concatenate_epochs([epochs['standard'][range(40)],
                                          epochs['standard'][182:222]])
epochs_standard.load_data()  # Resampling to save memory.
epochs_standard.resample(600, npad='auto')
epochs_deviant = epochs['deviant'].load_data()
epochs_deviant.resample(600, npad='auto')
del epochs, picks


I am working on the file "0.12/_downloads/plot_channel_epochs_image.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'
event_id, tmin, tmax = 1, -0.2, 0.5

# Setup for reading the raw data
raw = io.read_raw_fif(raw_fname)
events = mne.read_events(event_fname)

# Set up pick list: EEG + MEG - bad channels (modify to your needs)
raw.info['bads'] = ['MEG 2443', 'EEG 053']
picks = mne.pick_types(raw.info, meg='grad', eeg=False, stim=True, eog=True,
                       exclude='bads')

# Read epochs
epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,
                    picks=picks, baseline=(None, 0), preload=True,
                    reject=dict(grad=4000e-13, eog=150e-6))


I am working on the file "0.12/_downloads/plot_channel_epochs_image.ipynb".
The first task is:
Show event related fields images
Can you write Python code for it?

# and order with spectral reordering
# If you don't have scikit-learn installed set order_func to None
from sklearn.cluster.spectral import spectral_embedding  # noqa
from sklearn.metrics.pairwise import rbf_kernel   # noqa

def order_func(times, data):
    this_data = data[:, (times > 0.0) & (times < 0.350)]
    this_data /= np.sqrt(np.sum(this_data ** 2, axis=1))[:, np.newaxis]
    return np.argsort(spectral_embedding(rbf_kernel(this_data, gamma=1.),
                      n_components=1, random_state=0).ravel())

good_pick = 97  # channel with a clear evoked response
bad_pick = 98  # channel with no evoked response

plt.close('all')
mne.viz.plot_epochs_image(epochs, [good_pick, bad_pick], sigma=0.5, vmin=-100,
                          vmax=250, colorbar=True, order=order_func, show=True)


I am working on the file "0.12/_downloads/plot_channel_epochs_image.ipynb".
The first task is:
Visualize channel over epochs as an image
This will produce what is sometimes called an event related
potential / field (ERP/ERF) image.
2 images are produced. One with a good channel and one with a channel
that does not see any evoked field.
It is also demonstrated how to reorder the epochs using a 1d spectral
embedding as described in:
Graph-based variability estimation in single-trial event-related neural
responses A. Gramfort, R. Keriven, M. Clerc, 2010,
Biomedical Engineering, IEEE Trans. on, vol. 57 (5), 1051-1061
https://hal.inria.fr/inria-00497023
Can you write Python code for it?

# Authors: Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
#
# License: BSD (3-clause)

import numpy as np
import matplotlib.pyplot as plt

import mne
from mne import io
from mne.datasets import sample

print(__doc__)

data_path = sample.data_path()


I am working on the file "0.12/_downloads/plot_clickable_image.ipynb".
The first task is:
================================================================
Demonstration of how to use ClickableImage / generate_2d_layout.
================================================================
In this example, we open an image file, then use ClickableImage to
return 2D locations of mouse clicks (or load a file already created).
Then, we use generate_2d_layout to turn those xy positions into a layout
for use with plotting topo maps. In this way, you can take arbitrary xy
positions and turn them into a plottable layout.
Can you write Python code for it?

# Authors: Christopher Holdgraf <choldgraf@berkeley.edu>
#
# License: BSD (3-clause)
from scipy.ndimage import imread
import numpy as np
from matplotlib import pyplot as plt
from os import path as op
import mne
from mne.viz import ClickableImage, add_background_image  # noqa
from mne.channels import generate_2d_layout  # noqa

print(__doc__)

# Set parameters and paths
plt.rcParams['image.cmap'] = 'gray'

im_path = op.join(op.dirname(mne.__file__), 'data', 'image', 'mni_brain.gif')
# We've already clicked and exported
layout_path = op.join(op.dirname(mne.__file__), 'data', 'image')
layout_name = 'custom_layout.lout'


I am working on the file "0.12/_downloads/plot_clickable_image.ipynb".
The first task is:
Load data and click
Can you write Python code for it?

im = imread(im_path)
plt.imshow(im)
"""
This code opens the image so you can click on it. Commented out
because we've stored the clicks as a layout file already.

# The click coordinates are stored as a list of tuples
click = ClickableImage(im)
click.plot_clicks()
coords = click.coords

# Generate a layout from our clicks and normalize by the image
lt = generate_2d_layout(np.vstack(coords), bg_image=im)
lt.save(layout_path + layout_name)  # To save if we want
"""
# We've already got the layout, load it
lt = mne.channels.read_layout(layout_name, path=layout_path, scale=False)

# Create some fake data
nchans = len(lt.pos)
nepochs = 50
sr = 1000
nsec = 5
events = np.arange(nepochs).reshape([-1, 1])
events = np.hstack([events, np.zeros([nepochs, 2], dtype=int)])
data = np.random.randn(nepochs, nchans, sr * nsec)
info = mne.create_info(nchans, sr, ch_types='eeg')
epochs = mne.EpochsArray(data, info, events)
evoked = epochs.average()

# Using the native plot_topo function with the image plotted in the background
f = evoked.plot_topo(layout=lt, fig_background=im)


I am working on the file "0.12/_downloads/plot_cluster_stats_evoked.ipynb".
The first task is:
Compute statistic
Can you write Python code for it?

threshold = 6.0
T_obs, clusters, cluster_p_values, H0 = \
    permutation_cluster_test([condition1, condition2], n_permutations=1000,
                             threshold=threshold, tail=1, n_jobs=1)


I am working on the file "0.12/_downloads/plot_cluster_stats_evoked.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'
event_id = 1
tmin = -0.2
tmax = 0.5

#   Setup for reading the raw data
raw = io.read_raw_fif(raw_fname)
events = mne.read_events(event_fname)

channel = 'MEG 1332'  # include only this channel in analysis
include = [channel]


I am working on the file "0.12/_downloads/plot_compute_covariance.ipynb".
The first task is:
.. _tut_compute_covariance:
Computing covariance matrix
Can you write Python code for it?

import os.path as op

import mne
from mne.datasets import sample


I am working on the file "0.12/_downloads/plot_compute_covariance.ipynb".
The first task is:
Note that this method also attenuates the resting state activity in your
source estimates.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_compute_covariance.ipynb".
The first task is:
Now that you the covariance matrix in a python object you can save it to a
file with :func:mne.write_cov. Later you can read it back to a python
object using :func:mne.read_cov.
You can also use the pre-stimulus baseline to estimate the noise covariance.
First we have to construct the epochs. When computing the covariance, you
should use baseline correction when constructing the epochs. Otherwise the
covariance matrix will be inaccurate. In MNE this is done by default, but
just to be sure, we define it here manually.
Can you write Python code for it?

events = mne.find_events(raw)
epochs = mne.Epochs(raw, events, event_id=1, tmin=-0.2, tmax=0.0,
                    baseline=(-0.2, 0.0))


I am working on the file "0.12/_downloads/plot_compute_covariance.ipynb".
The first task is:
Plot the covariance matrices
Try setting proj to False to see the effect. Notice that the projectors in
epochs are already applied, so proj parameter has no effect.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_compute_covariance.ipynb".
The first task is:
Source estimation method such as MNE require a noise estimations from the
recordings. In this tutorial we cover the basics of noise covariance and
construct a noise covariance matrix that can be used when computing the
inverse solution. For more information, see :ref:BABDEEEB.
Can you write Python code for it?

data_path = sample.data_path()
raw_empty_room_fname = op.join(
    data_path, 'MEG', 'sample', 'ernoise_raw.fif')
raw_empty_room = mne.io.read_raw_fif(raw_empty_room_fname)
raw_fname = op.join(data_path, 'MEG', 'sample', 'sample_audvis_raw.fif')
raw = mne.io.read_raw_fif(raw_fname)
raw.info['bads'] += ['EEG 053']  # bads + 1 more


I am working on the file "0.12/_downloads/plot_compute_covariance.ipynb".
The first task is:
This plot displays both, the whitened evoked signals for each channels and
the whitened GFP. The numbers in the GFP panel represent the estimated rank
of the data, which amounts to the effective degrees of freedom by which the
squared sum across sensors is divided when computing the whitened GFP.
The whitened GFP also helps detecting spurious late evoked components which
can be the consequence of over- or under-regularization.
Note that if data have been processed using signal space separation
(SSS) [2],
gradiometers and magnetometers will be displayed jointly because both are
reconstructed from the same SSS basis vectors with the same numerical rank.
This also implies that both sensor types are not any longer statistically
independent.
These methods for evaluation can be used to assess model violations.
Additional
introductory materials can be found here <https://goo.gl/ElWrxe>.
For expert use cases or debugging the alternative estimators can also be
compared:
Can you write Python code for it?

covs = mne.compute_covariance(epochs, tmax=0., method=('empirical', 'shrunk'),
                              return_estimators=True)
evoked = epochs.average()
evoked.plot_white(covs)


I am working on the file "0.12/_downloads/plot_compute_mne_inverse_epochs_in_label.ipynb".
The first task is:
Viewing single trial dSPM and average dSPM for unflipped pooling over label
Compare to (1) Inverse (dSPM) then average, (2) Evoked then dSPM
Can you write Python code for it?

# Single trial
plt.figure()
for k, stc_trial in enumerate(stcs):
    plt.plot(times, np.mean(stc_trial.data, axis=0).T, 'k--',
             label='Single Trials' if k == 0 else '_nolegend_',
             alpha=0.5)

# Single trial inverse then average.. making linewidth large to not be masked
plt.plot(times, label_mean, 'b', linewidth=6,
         label='dSPM first, then average')

# Evoked and then inverse
plt.plot(times, label_mean_evoked, 'r', linewidth=2,
         label='Average first, then dSPM')

plt.xlabel('time (ms)')
plt.ylabel('dSPM value')
plt.legend()
plt.show()


I am working on the file "0.12/_downloads/plot_compute_mne_inverse_raw_in_label.ipynb".
The first task is:
Compute sLORETA inverse solution on raw data
Compute sLORETA inverse solution on raw dataset restricted
to a brain label and stores the solution in stc files for
visualisation.
Can you write Python code for it?

# Author: Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
#
# License: BSD (3-clause)

import matplotlib.pyplot as plt

import mne
from mne.datasets import sample
from mne.minimum_norm import apply_inverse_raw, read_inverse_operator

print(__doc__)

data_path = sample.data_path()
fname_inv = data_path + '/MEG/sample/sample_audvis-meg-oct-6-meg-inv.fif'
fname_raw = data_path + '/MEG/sample/sample_audvis_raw.fif'
label_name = 'Aud-lh'
fname_label = data_path + '/MEG/sample/labels/%s.label' % label_name

snr = 1.0  # use smaller SNR for raw data
lambda2 = 1.0 / snr ** 2
method = "sLORETA"  # use sLORETA method (could also be MNE or dSPM)

# Load data
raw = mne.io.read_raw_fif(fname_raw)
inverse_operator = read_inverse_operator(fname_inv)
label = mne.read_label(fname_label)

start, stop = raw.time_as_index([0, 15])  # read the first 15s of data

# Compute inverse solution
stc = apply_inverse_raw(raw, inverse_operator, lambda2, method, label,
                        start, stop, pick_ori=None)

# Save result in stc files
stc.save('mne_%s_raw_inverse_%s' % (method, label_name))


I am working on the file "0.12/_downloads/plot_compute_raw_data_spectrum.ipynb".
The first task is:
Compute the power spectral density of raw data
This script shows how to compute the power spectral density (PSD)
of measurements on a raw dataset. It also show the effect of applying SSP
to the data to reduce ECG and EOG artifacts.
Can you write Python code for it?

# Authors: Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
#          Martin Luessi <mluessi@nmr.mgh.harvard.edu>
#          Eric Larson <larson.eric.d@gmail.com>
# License: BSD (3-clause)

import numpy as np
import matplotlib.pyplot as plt

import mne
from mne import io, read_proj, read_selection
from mne.datasets import sample
from mne.time_frequency import psd_multitaper

print(__doc__)


I am working on the file "0.12/_downloads/plot_covariance_whitening_dspm.ipynb".
The first task is:
Estimate covariances
Can you write Python code for it?

samples_epochs = 5, 15,
method = 'empirical', 'shrunk'
colors = 'steelblue', 'red'

evokeds = list()
stcs = list()
methods_ordered = list()
for n_train in samples_epochs:
    # estimate covs based on a subset of samples
    # make sure we have the same number of conditions.
    events_ = np.concatenate([events[events[:, 2] == id_][:n_train]
                              for id_ in [event_ids[k] for k in conditions]])
    epochs_train = mne.Epochs(raw, events_, event_ids, tmin, tmax, picks=picks,
                              baseline=baseline, preload=True, reject=reject)
    epochs_train.equalize_event_counts(event_ids, copy=False)
    assert len(epochs_train) == 2 * n_train

    noise_covs = compute_covariance(
        epochs_train, method=method, tmin=None, tmax=0,  # baseline only
        return_estimators=True)  # returns list
    # prepare contrast
    evokeds = [epochs_train[k].average() for k in conditions]
    del epochs_train, events_
    # do contrast

    # We skip empirical rank estimation that we introduced in response to
    # the findings in reference [1] to use the naive code path that
    # triggered the behavior described in [1]. The expected true rank is
    # 274 for this dataset. Please do not do this with your data but
    # rely on the default rank estimator that helps regularizing the
    # covariance.
    stcs.append(list())
    methods_ordered.append(list())
    for cov in noise_covs:
        inverse_operator = make_inverse_operator(evokeds[0].info, forward,
                                                 cov, loose=0.2, depth=0.8,
                                                 rank=274)
        stc_a, stc_b = (apply_inverse(e, inverse_operator, lambda2, "dSPM",
                                      pick_ori=None) for e in evokeds)
        stc = stc_a - stc_b
        methods_ordered[-1].append(cov['method'])
        stcs[-1].append(stc)
    del inverse_operator, evokeds, cov, noise_covs, stc, stc_a, stc_b
del raw, forward  # save some memory


I am working on the file "0.12/_downloads/plot_covariance_whitening_dspm.ipynb".
The first task is:
Get data
Can you write Python code for it?

data_path = spm_face.data_path()
subjects_dir = data_path + '/subjects'

raw_fname = data_path + '/MEG/spm/SPM_CTF_MEG_example_faces%d_3D.ds'

raw = io.read_raw_ctf(raw_fname % 1)  # Take first run
# To save time and memory for this demo, we'll just use the first
# 2.5 minutes (all we need to get 30 total events) and heavily
# resample 480->60 Hz (usually you wouldn't do either of these!)
raw = raw.crop(0, 150.).load_data().resample(60, npad='auto')

picks = mne.pick_types(raw.info, meg=True, exclude='bads')
raw.filter(1, None, method='iir', n_jobs=1)

events = mne.find_events(raw, stim_channel='UPPT001')

event_ids = {"faces": 1, "scrambled": 2}
tmin, tmax = -0.2, 0.5
baseline = None  # no baseline as high-pass is applied
reject = dict(mag=3e-12)

# Make source space

trans = data_path + '/MEG/spm/SPM_CTF_MEG_example_faces1_3D_raw-trans.fif'
src = mne.setup_source_space('spm', fname=None, spacing='oct6',
                             subjects_dir=subjects_dir, add_dist=False)
bem = data_path + '/subjects/spm/bem/spm-5120-5120-5120-bem-sol.fif'
forward = mne.make_forward_solution(raw.info, trans, src, bem)
forward = mne.convert_forward_solution(forward, surf_ori=True)
del src

# inverse parameters
conditions = 'faces', 'scrambled'
snr = 3.0
lambda2 = 1.0 / snr ** 2
method = 'dSPM'
clim = dict(kind='value', lims=[0, 2.5, 5])


I am working on the file "0.12/_downloads/plot_creating_data_structures.ipynb".
The first task is:
Creating :class:Evoked <mne.Evoked> Objects
If you already have data that is collapsed across trials, you may also
directly create an evoked array.  Its constructor accepts an array of
shape(n_chans, n_times) in addition to some bookkeeping parameters.
Can you write Python code for it?

# The averaged data
data_evoked = data.mean(0)

# The number of epochs that were averaged
nave = data.shape[0]

# A comment to describe to evoked (usually the condition name)
comment = "Smiley faces"

# Create the Evoked object
evoked_array = mne.EvokedArray(data_evoked, info, tmin,
                               comment=comment, nave=nave)
print(evoked_array)
_ = evoked_array.plot()


I am working on the file "0.12/_downloads/plot_creating_data_structures.ipynb".
The first task is:
Creating :class:Info <mne.Info> objects
.. note:: for full documentation on the Info object, see
          :ref:tut_info_objects. See also
          :ref:sphx_glr_auto_examples_io_plot_objects_from_arrays.py.
Normally, :class:mne.Info objects are created by the various
:ref:data import functions <ch_convert>.
However, if you wish to create one from scratch, you can use the
:func:mne.create_info function to initialize the minimally required
fields. Further fields can be assigned later as one would with a regular
dictionary.
The following creates the absolute minimum info structure:
Can you write Python code for it?

# Create some dummy metadata
n_channels = 32
sampling_rate = 200
info = mne.create_info(32, sampling_rate)
print(info)


I am working on the file "0.12/_downloads/plot_creating_data_structures.ipynb".
The first task is:
Finally, we must specify the beginning of an epoch (the end will be inferred
from the sampling frequency and n_samples)
Can you write Python code for it?

# Trials were cut from -0.1 to 1.0 seconds
tmin = -0.1


I am working on the file "0.12/_downloads/plot_creating_data_structures.ipynb".
The first task is:
It is necessary to supply an "events" array in order to create an Epochs
object. This is of shape(n_events, 3) where the first column is the index
of the event, the second column is the length of the event, and the third
column is the event type.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_creating_data_structures.ipynb".
The first task is:
More information about the event codes: subject was either smiling or
frowning
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_creating_data_structures.ipynb".
The first task is:
Now we can create the :class:mne.EpochsArray object
Can you write Python code for it?

custom_epochs = mne.EpochsArray(data, info, events, tmin, event_id)

print(custom_epochs)

# We can treat the epochs object as we would any other
_ = custom_epochs['smiling'].average().plot()


I am working on the file "0.12/_downloads/plot_creating_data_structures.ipynb".
The first task is:
You can also supply more extensive metadata:
Can you write Python code for it?

# Names for each channel
channel_names = ['MEG1', 'MEG2', 'Cz', 'Pz', 'EOG']

# The type (mag, grad, eeg, eog, misc, ...) of each channel
channel_types = ['grad', 'grad', 'eeg', 'eeg', 'eog']

# The sampling rate of the recording
sfreq = 1000  # in Hertz

# The EEG channels use the standard naming strategy.
# By supplying the 'montage' parameter, approximate locations
# will be added for them
montage = 'standard_1005'

# Initialize required fields
info = mne.create_info(channel_names, sfreq, channel_types, montage)

# Add some more information
info['description'] = 'My custom dataset'
info['bads'] = ['Pz']  # Names of bad channels

print(info)


I am working on the file "0.12/_downloads/plot_cwt_sensor_connectivity.ipynb".
The first task is:
Compute seed based time-frequency connectivity in sensor space
Computes the connectivity between a seed-gradiometer close to the visual cortex
and all other gradiometers. The connectivity is computed in the time-frequency
domain using Morlet wavelets and the debiased Squared Weighted Phase Lag Index
[1] is used as connectivity metric.
[1] Vinck et al. "An improved index of phase-synchronization for electro-
    physiological data in the presence of volume-conduction, noise and
    sample-size bias" NeuroImage, vol. 55, no. 4, pp. 1548-1565, Apr. 2011.
Can you write Python code for it?

# Author: Martin Luessi <mluessi@nmr.mgh.harvard.edu>
#
# License: BSD (3-clause)

import numpy as np

import mne
from mne import io
from mne.connectivity import spectral_connectivity, seed_target_indices
from mne.datasets import sample
from mne.time_frequency import AverageTFR

print(__doc__)


I am working on the file "0.12/_downloads/plot_decoding_csp_eeg.ipynb".
The first task is:
Classification with linear discrimant analysis
Can you write Python code for it?

from sklearn.lda import LDA  # noqa
from sklearn.cross_validation import ShuffleSplit  # noqa

# Assemble a classifier
svc = LDA()
csp = CSP(n_components=4, reg=None, log=True)

# Define a monte-carlo cross-validation generator (reduce variance):
cv = ShuffleSplit(len(labels), 10, test_size=0.2, random_state=42)
scores = []
epochs_data = epochs.get_data()
epochs_data_train = epochs_train.get_data()

# Use scikit-learn Pipeline with cross_val_score function
from sklearn.pipeline import Pipeline  # noqa
from sklearn.cross_validation import cross_val_score  # noqa
clf = Pipeline([('CSP', csp), ('SVC', svc)])
scores = cross_val_score(clf, epochs_data_train, labels, cv=cv, n_jobs=1)

# Printing the results
class_balance = np.mean(labels == labels[0])
class_balance = max(class_balance, 1. - class_balance)
print("Classification accuracy: %f / Chance level: %f" % (np.mean(scores),
                                                          class_balance))

# plot CSP patterns estimated on full data for visualization
csp.fit_transform(epochs_data, labels)

evoked = epochs.average()
evoked.data = csp.patterns_.T
evoked.times = np.arange(evoked.data.shape[0])

layout = read_layout('EEG1005')
evoked.plot_topomap(times=[0, 1, 2, 3, 4, 5], ch_type='eeg', layout=layout,
                    scale_time=1, time_format='%i', scale=1,
                    unit='Patterns (AU)', size=1.5)


I am working on the file "0.12/_downloads/plot_decoding_csp_space.ipynb".
The first task is:
====================================================================
Decoding in sensor space data using the Common Spatial Pattern (CSP)
====================================================================
Decoding applied to MEG data in sensor space decomposed using CSP.
Here the classifier is applied to features extracted on CSP filtered signals.
See http://en.wikipedia.org/wiki/Common_spatial_pattern and [1]
[1] Zoltan J. Koles. The quantitative extraction and topographic mapping
    of the abnormal components in the clinical EEG. Electroencephalography
    and Clinical Neurophysiology, 79(6):440--447, December 1991.
Can you write Python code for it?

# Authors: Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
#          Romain Trachel <romain.trachel@inria.fr>
#
# License: BSD (3-clause)

import numpy as np
import matplotlib.pyplot as plt

import mne
from mne import io
from mne.datasets import sample

print(__doc__)

data_path = sample.data_path()


I am working on the file "0.12/_downloads/plot_decoding_csp_space.ipynb".
The first task is:
Decoding in sensor space using a linear SVM
Can you write Python code for it?

from sklearn.svm import SVC  # noqa
from sklearn.cross_validation import ShuffleSplit  # noqa
from mne.decoding import CSP  # noqa

n_components = 3  # pick some components
svc = SVC(C=1, kernel='linear')
csp = CSP(n_components=n_components)

# Define a monte-carlo cross-validation generator (reduce variance):
cv = ShuffleSplit(len(labels), 10, test_size=0.2, random_state=42)
scores = []
epochs_data = epochs.get_data()

for train_idx, test_idx in cv:
    y_train, y_test = labels[train_idx], labels[test_idx]

    X_train = csp.fit_transform(epochs_data[train_idx], y_train)
    X_test = csp.transform(epochs_data[test_idx])

    # fit classifier
    svc.fit(X_train, y_train)

    scores.append(svc.score(X_test, y_test))

# Printing the results
class_balance = np.mean(labels == labels[0])
class_balance = max(class_balance, 1. - class_balance)
print("Classification accuracy: %f / Chance level: %f" % (np.mean(scores),
                                                          class_balance))

# Or use much more convenient scikit-learn cross_val_score function using
# a Pipeline
from sklearn.pipeline import Pipeline  # noqa
from sklearn.cross_validation import cross_val_score  # noqa
cv = ShuffleSplit(len(labels), 10, test_size=0.2, random_state=42)
clf = Pipeline([('CSP', csp), ('SVC', svc)])
scores = cross_val_score(clf, epochs_data, labels, cv=cv, n_jobs=1)
print(scores.mean())  # should match results above

# And using reuglarized csp with Ledoit-Wolf estimator
csp = CSP(n_components=n_components, reg='ledoit_wolf')
clf = Pipeline([('CSP', csp), ('SVC', svc)])
scores = cross_val_score(clf, epochs_data, labels, cv=cv, n_jobs=1)
print(scores.mean())  # should get better results than above

# plot CSP patterns estimated on full data for visualization
csp.fit_transform(epochs_data, labels)
data = csp.patterns_
fig, axes = plt.subplots(1, 4)
for idx in range(4):
    mne.viz.plot_topomap(data[idx], evoked.info, axes=axes[idx], show=False)
fig.suptitle('CSP patterns')
fig.tight_layout()
fig.show()


I am working on the file "0.12/_downloads/plot_decoding_csp_space.ipynb".
The first task is:
Set parameters and read data
Can you write Python code for it?

raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'
tmin, tmax = -0.2, 0.5
event_id = dict(aud_l=1, vis_l=3)

# Setup for reading the raw data
raw = io.read_raw_fif(raw_fname, preload=True)
raw.filter(2, None, method='iir')  # replace baselining with high-pass
events = mne.read_events(event_fname)

raw.info['bads'] = ['MEG 2443']  # set bad channels
picks = mne.pick_types(raw.info, meg='grad', eeg=False, stim=False, eog=False,
                       exclude='bads')

# Read epochs
epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,
                    picks=picks, baseline=None, preload=True)

labels = epochs.events[:, -1]
evoked = epochs.average()


I am working on the file "0.12/_downloads/plot_decoding_spatio_temporal_source.ipynb".
The first task is:
Decoding in sensor space using a linear SVM
Can you write Python code for it?

# Make arrays X and y such that :
# X is 3d with X.shape[0] is the total number of epochs to classify
# y is filled with integers coding for the class to predict
# We must have X.shape[0] equal to y.shape[0]

# we know the first half belongs to the first class, the second one
y = np.repeat([0, 1], len(X) / 2)   # belongs to the second class
X = X.reshape(n_epochs, n_vertices * n_times)
# we have to normalize the data before supplying them to our classifier
X -= X.mean(axis=0)
X /= X.std(axis=0)

# prepare classifier
from sklearn.svm import SVC  # noqa
from sklearn.cross_validation import ShuffleSplit  # noqa

# Define a monte-carlo cross-validation generator (reduce variance):
n_splits = 10
clf = SVC(C=1, kernel='linear')
cv = ShuffleSplit(len(X), n_splits, test_size=0.2)

# setup feature selection and classification pipeline
from sklearn.feature_selection import SelectKBest, f_classif  # noqa
from sklearn.pipeline import Pipeline  # noqa

# we will use an ANOVA f-test to preselect relevant spatio-temporal units
feature_selection = SelectKBest(f_classif, k=500)  # take the best 500
# to make life easier we will create a pipeline object
anova_svc = Pipeline([('anova', feature_selection), ('svc', clf)])

# initialize score and feature weights result arrays
scores = np.zeros(n_splits)
feature_weights = np.zeros([n_vertices, n_times])

# hold on, this may take a moment
for ii, (train, test) in enumerate(cv):
    anova_svc.fit(X[train], y[train])
    y_pred = anova_svc.predict(X[test])
    y_test = y[test]
    scores[ii] = np.sum(y_pred == y_test) / float(len(y_test))
    feature_weights += feature_selection.inverse_transform(clf.coef_) \
        .reshape(n_vertices, n_times)

print('Average prediction accuracy: %0.3f | standard deviation:  %0.3f'
      % (scores.mean(), scores.std()))

# prepare feature weights for visualization
feature_weights /= (ii + 1)  # create average weights
# create mask to avoid division error
feature_weights = np.ma.masked_array(feature_weights, feature_weights == 0)
# normalize scores...


I am working on the file "0.12/_downloads/plot_decoding_spatio_temporal_source.ipynb".
The first task is:
Decoding source space data
Decoding, a.k.a MVPA or supervised machine learning applied to MEG
data in source space on the left cortical surface. Here f-test feature
selection is employed to confine the classification to the potentially
relevant features. The classifier then is trained to selected features of
epochs in source space.
Can you write Python code for it?

# Author: Denis A. Engemann <denis.engemann@gmail.com>
#         Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
#
# License: BSD (3-clause)

import mne
import os
import numpy as np
from mne import io
from mne.datasets import sample
from mne.minimum_norm import apply_inverse_epochs, read_inverse_operator

print(__doc__)

data_path = sample.data_path()
fname_fwd = data_path + 'MEG/sample/sample_audvis-meg-oct-6-fwd.fif'
fname_evoked = data_path + '/MEG/sample/sample_audvis-ave.fif'
subjects_dir = data_path + '/subjects'
subject = os.environ['SUBJECT'] = subjects_dir + '/sample'
os.environ['SUBJECTS_DIR'] = subjects_dir


I am working on the file "0.12/_downloads/plot_decoding_spatio_temporal_source.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'
fname_cov = data_path + '/MEG/sample/sample_audvis-cov.fif'
label_names = 'Aud-rh', 'Vis-rh'
fname_inv = data_path + '/MEG/sample/sample_audvis-meg-oct-6-meg-inv.fif'

tmin, tmax = -0.2, 0.5
event_id = dict(aud_r=2, vis_r=4)  # load contra-lateral conditions

# Setup for reading the raw data
raw = io.read_raw_fif(raw_fname, preload=True)
raw.filter(2, None, method='iir')  # replace baselining with high-pass
events = mne.read_events(event_fname)

# Set up pick list: MEG - bad channels (modify to your needs)
raw.info['bads'] += ['MEG 2443']  # mark bads
picks = mne.pick_types(raw.info, meg=True, eeg=False, stim=True, eog=True,
                       exclude='bads')

# Read epochs
epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,
                    picks=picks, baseline=None, preload=True,
                    reject=dict(grad=4000e-13, eog=150e-6),
                    decim=5)  # decimate to save memory and increase speed

epochs.equalize_event_counts(list(event_id.keys()), 'mintime', copy=False)
epochs_list = [epochs[k] for k in event_id]

# Compute inverse solution
snr = 3.0
lambda2 = 1.0 / snr ** 2
method = "dSPM"  # use dSPM method (could also be MNE or sLORETA)
n_times = len(epochs.times)
n_vertices = 3732
n_epochs = len(epochs.events)

# Load data and compute inverse solution and stcs for each epoch.

noise_cov = mne.read_cov(fname_cov)
inverse_operator = read_inverse_operator(fname_inv)
X = np.zeros([n_epochs, n_vertices, n_times])

# to save memory, we'll load and transform our epochs step by step.
for condition_count, ep in zip([0, n_epochs / 2], epochs_list):
    stcs = apply_inverse_epochs(ep, inverse_operator, lambda2,
                                method, pick_ori="normal",  # saves us memory
                                return_generator=True)
    for jj, stc in enumerate(stcs):
        X[condition_count + jj] = stc.lh_data


I am working on the file "0.12/_downloads/plot_decoding_xdawn_eeg.ipynb".
The first task is:
Set parameters and read data
Can you write Python code for it?

raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'
tmin, tmax = -0.1, 0.3
event_id = dict(aud_l=1, aud_r=2, vis_l=3, vis_r=4)

# Setup for reading the raw data
raw = io.read_raw_fif(raw_fname, preload=True)
raw.filter(1, 20, method='iir')
events = read_events(event_fname)

picks = pick_types(raw.info, meg=False, eeg=True, stim=False, eog=False,
                   exclude='bads')

epochs = Epochs(raw, events, event_id, tmin, tmax, proj=False,
                picks=picks, baseline=None, preload=True,
                add_eeg_ref=False, verbose=False)

# Create classification pipeline
clf = make_pipeline(Xdawn(n_components=3),
                    EpochsVectorizer(),
                    MinMaxScaler(),
                    LogisticRegression(penalty='l1'))

# Get the labels
labels = epochs.events[:, -1]

# Cross validator
cv = StratifiedKFold(y=labels, n_folds=10, shuffle=True, random_state=42)

# Do cross-validation
preds = np.empty(len(labels))
for train, test in cv:
    clf.fit(epochs[train], labels[train])
    preds[test] = clf.predict(epochs[test])

# Classification report
target_names = ['aud_l', 'aud_r', 'vis_l', 'vis_r']
report = classification_report(labels, preds, target_names=target_names)
print(report)

# Normalized confusion matrix
cm = confusion_matrix(labels, preds)
cm_normalized = cm.astype(float) / cm.sum(axis=1)[:, np.newaxis]

# Plot confusion matrix
plt.imshow(cm_normalized, interpolation='nearest', cmap=plt.cm.Blues)
plt.title('Normalized Confusion matrix')
plt.colorbar()
tick_marks = np.arange(len(target_names))
plt.xticks(tick_marks, target_names, rotation=45)
plt.yticks(tick_marks, target_names)
tight_layout()
plt.ylabel('True label')
plt.xlabel('Predicted label')
plt.show()


I am working on the file "0.12/_downloads/plot_decoding_xdawn_eeg.ipynb".
The first task is:
XDAWN Decoding From EEG data
ERP decoding with Xdawn. For each event type, a set of spatial Xdawn filters
are trained and applied on the signal. Channels are concatenated and rescaled
to create features vectors that will be fed into a Logistic Regression.
References
[1] Rivet, B., Souloumiac, A., Attina, V., & Gibert, G. (2009). xDAWN
algorithm to enhance evoked potentials: application to brain-computer
interface. Biomedical Engineering, IEEE Transactions on, 56(8), 2035-2043.
[2] Rivet, B., Cecotti, H., Souloumiac, A., Maby, E., & Mattout, J. (2011,
August). Theoretical analysis of xDAWN algorithm: application to an
efficient sensor selection in a P300 BCI. In Signal Processing Conference,
2011 19th European (pp. 1382-1386). IEEE.
Can you write Python code for it?

# Authors: Alexandre Barachant <alexandre.barachant@gmail.com>
#
# License: BSD (3-clause)

import numpy as np
import matplotlib.pyplot as plt

from sklearn.cross_validation import StratifiedKFold
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.preprocessing import MinMaxScaler

from mne import io, pick_types, read_events, Epochs
from mne.datasets import sample
from mne.preprocessing import Xdawn
from mne.decoding import EpochsVectorizer
from mne.viz import tight_layout

print(__doc__)

data_path = sample.data_path()


I am working on the file "0.12/_downloads/plot_define_target_events.ipynb".
The first task is:
Find stimulus event followed by quick button presses
Can you write Python code for it?

reference_id = 5  # presentation of a smiley face
target_id = 32  # button press
sfreq = raw.info['sfreq']  # sampling rate
tmin = 0.1  # trials leading to very early responses will be rejected
tmax = 0.59  # ignore face stimuli followed by button press later than 590 ms
new_id = 42  # the new event id for a hit. If None, reference_id is used.
fill_na = 99  # the fill value for misses

events_, lag = define_target_events(events, reference_id, target_id,
                                    sfreq, tmin, tmax, new_id, fill_na)

print(events_)  # The 99 indicates missing or too late button presses

# besides the events also the lag between target and reference is returned
# this could e.g. be used as parametric regressor in subsequent analyses.

print(lag[lag != fill_na])  # lag in milliseconds

# #############################################################################
# Construct epochs

tmin_ = -0.2
tmax_ = 0.4
event_id = dict(early=new_id, late=fill_na)

epochs = mne.Epochs(raw, events_, event_id, tmin_,
                    tmax_, picks=picks, baseline=(None, 0),
                    reject=dict(mag=4e-12))

# average epochs and get an Evoked dataset.

early, late = [epochs[k].average() for k in event_id]


I am working on the file "0.12/_downloads/plot_define_target_events.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'

#   Setup for reading the raw data
raw = io.read_raw_fif(raw_fname)
events = mne.read_events(event_fname)

#   Set up pick list: EEG + STI 014 - bad channels (modify to your needs)
include = []  # or stim channels ['STI 014']
raw.info['bads'] += ['EEG 053']  # bads

# pick MEG channels
picks = mne.pick_types(raw.info, meg='mag', eeg=False, stim=False, eog=True,
                       include=include, exclude='bads')


I am working on the file "0.12/_downloads/plot_define_target_events.ipynb".
The first task is:
View evoked response
Can you write Python code for it?

times = 1e3 * epochs.times  # time in milliseconds
title = 'Evoked response followed by %s button press'

plt.clf()
ax = plt.subplot(2, 1, 1)
early.plot(axes=ax)
plt.title(title % 'late')
plt.ylabel('Evoked field (fT)')
ax = plt.subplot(2, 1, 2)
late.plot(axes=ax)
plt.title(title % 'early')
plt.ylabel('Evoked field (fT)')
plt.show()


I am working on the file "0.12/_downloads/plot_dics_source_power.ipynb".
The first task is:
Compute source power using DICS beamfomer
Compute a Dynamic Imaging of Coherent Sources (DICS) filter from single trial
activity to estimate source power for two frequencies of interest.
The original reference for DICS is:
Gross et al. Dynamic imaging of coherent sources: Studying neural interactions
in the human brain. PNAS (2001) vol. 98 (2) pp. 694-699
Can you write Python code for it?

# Author: Roman Goj <roman.goj@gmail.com>
#         Denis Engemann <denis.engemann@gmail.com>
#
# License: BSD (3-clause)

import mne
from mne.datasets import sample
from mne.time_frequency import compute_epochs_csd
from mne.beamformer import dics_source_power

print(__doc__)

data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_raw-eve.fif'
fname_fwd = data_path + '/MEG/sample/sample_audvis-meg-eeg-oct-6-fwd.fif'
subjects_dir = data_path + '/subjects'


I am working on the file "0.12/_downloads/plot_dics_source_power.ipynb".
The first task is:
Read raw data
Can you write Python code for it?

raw = mne.io.read_raw_fif(raw_fname)
raw.info['bads'] = ['MEG 2443']  # 1 bad MEG channel

# Set picks
picks = mne.pick_types(raw.info, meg=True, eeg=False, eog=False,
                       stim=False, exclude='bads')

# Read epochs
event_id, tmin, tmax = 1, -0.2, 0.5
events = mne.read_events(event_fname)
epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,
                    picks=picks, baseline=(None, 0), preload=True,
                    reject=dict(grad=4000e-13, mag=4e-12))
evoked = epochs.average()

# Read forward operator
forward = mne.read_forward_solution(fname_fwd, surf_ori=True)

# Computing the data and noise cross-spectral density matrices
# The time-frequency window was chosen on the basis of spectrograms from
# example time_frequency/plot_time_frequency.py
# As fsum is False compute_epochs_csd returns a list of CrossSpectralDensity
# instances than can then be passed to dics_source_power
data_csds = compute_epochs_csd(epochs, mode='multitaper', tmin=0.04, tmax=0.15,
                               fmin=15, fmax=30, fsum=False)
noise_csds = compute_epochs_csd(epochs, mode='multitaper', tmin=-0.11,
                                tmax=-0.001, fmin=15, fmax=30, fsum=False)

# Compute DICS spatial filter and estimate source power
stc = dics_source_power(epochs.info, forward, noise_csds, data_csds)

clim = dict(kind='value', lims=[1.6, 1.9, 2.2])
for i, csd in enumerate(data_csds):
    message = 'DICS source power at %0.1f Hz' % csd.frequencies[0]
    brain = stc.plot(surface='inflated', hemi='rh', subjects_dir=subjects_dir,
                     time_label=message, figure=i, clim=clim)
    brain.set_data_time_index(i)
    brain.show_view('lateral')
    # Uncomment line below to save images
    # brain.save_image('DICS_source_power_freq_%d.png' % csd.frequencies[0])


I am working on the file "0.12/_downloads/plot_dipole_fit.ipynb".
The first task is:
Estimate the time course of a single dipole with fixed position and
orientation (the one that maximized GOF)over the entire interval
Can you write Python code for it?

dip_fixed = mne.fit_dipole(evoked_full, fname_cov, fname_bem, fname_trans,
                           pos=dip.pos[best_idx], ori=dip.ori[best_idx])[0]
dip_fixed.plot()


I am working on the file "0.12/_downloads/plot_dipole_fit.ipynb".
The first task is:
Let's localize the N100m (using MEG only)
Can you write Python code for it?

evoked = mne.read_evokeds(fname_ave, condition='Right Auditory',
                          baseline=(None, 0))
evoked.pick_types(meg=True, eeg=False)
evoked_full = evoked.copy()
evoked.crop(0.07, 0.08)

# Fit a dipole
dip = mne.fit_dipole(evoked, fname_cov, fname_bem, fname_trans)[0]

# Plot the result in 3D brain
dip.plot_locations(fname_trans, 'sample', subjects_dir)


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
.. _tut_erp:
EEG processing and Event Related Potentials (ERPs)
For a generic introduction to the computation of ERP and ERF
see :ref:tut_epoching_and_averaging. Here we cover the specifics
of EEG, namely:
    - setting the reference
    - using standard montages :func:mne.channels.Montage
    - Evoked arithmetic (e.g. differences)
Can you write Python code for it?

import mne
from mne.datasets import sample


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
And it's actually possible to plot the channel locations using
the :func:mne.io.Raw.plot_sensors method
Can you write Python code for it?

raw.plot_sensors()
raw.plot_sensors('3d')  # in 3D


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
And to change the nameo of the EOG channel
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
Average reference: This is normally added by default, but can also
be added explicitly.
Can you write Python code for it?

raw_car, _ = mne.io.set_eeg_reference(raw)
evoked_car = mne.Epochs(raw_car, **epochs_params).average()
del raw_car  # save memory

title = 'EEG Average reference'
evoked_car.plot(titles=dict(eeg=title))
evoked_car.plot_topomap(times=[0.1], size=3., title=title)


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
By looking at the measurement info you will see that we have now
59 EEG channels and 1 EOG channel
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
Evoked arithmetics
Trial subsets from Epochs can be selected using 'tags' separated by '/'.
Evoked objects support basic arithmetic.
First, we create an Epochs object containing 4 conditions.
Can you write Python code for it?

event_id = {'left/auditory': 1, 'right/auditory': 2,
            'left/visual': 3, 'right/visual': 4}
epochs_params = dict(events=events, event_id=event_id, tmin=tmin, tmax=tmax,
                     reject=reject)
epochs = mne.Epochs(raw, **epochs_params)

print(epochs)


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
In practice it's quite common to have some EEG channels that are actually
EOG channels. To change a channel type you can use the
:func:mne.io.Raw.set_channel_types method. For example
to treat an EOG channel as EEG you can change its type using
Can you write Python code for it?

raw.set_channel_types(mapping={'EOG 061': 'eeg'})
print(raw.info)


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
Let's reset the EOG channel back to EOG type.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
Let's restrict the data to the EEG channels
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
Note that by default, this is a trial-weighted average. If you have
imbalanced trial numbers, consider either equalizing the number of events per
condition (using Epochs.equalize_event_counts), or the combine_evoked
function.
As an example, first, we create individual ERPs for each condition.
Can you write Python code for it?

aud_l = epochs["auditory", "left"].average()
aud_r = epochs["auditory", "right"].average()
vis_l = epochs["visual", "left"].average()
vis_r = epochs["visual", "right"].average()

all_evokeds = [aud_l, aud_r, vis_l, vis_r]

# This could have been much simplified with a list comprehension:
# all_evokeds = [epochs[cond] for cond in event_id]

# Then, we construct and plot an unweighted average of left vs. right trials.
mne.combine_evoked(all_evokeds, weights=(1, -1, 1, -1)).plot_joint()


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
Setting EEG montage
In the case where your data don't have locations you can set them
using a :func:mne.channels.Montage. MNE comes with a set of default
montages. To read one of them do:
Can you write Python code for it?

montage = mne.channels.read_montage('standard_1020')
print(montage)


I am working on the file "0.12/_downloads/plot_eeg_erp.ipynb".
The first task is:
The EEG channels in the sample dataset already have locations.
These locations are available in the 'loc' of each channel description.
For the first channel we get
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_epoching_and_averaging.ipynb".
The first task is:
.. _tut_epoching_and_averaging:
Epoching and averaging (ERP/ERF)
Can you write Python code for it?

import os.path as op
import numpy as np

import mne


I am working on the file "0.12/_downloads/plot_epoching_and_averaging.ipynb".
The first task is:
Finally, let's plot the evoked responses.
Can you write Python code for it?