I am working on the file "0.12/_downloads/plot_stats_cluster_methods.ipynb".
The first task is:
Visualize results
Can you write Python code for it?

fig = plt.figure(facecolor='w')

x, y = np.mgrid[0:width, 0:width]
kwargs = dict(rstride=1, cstride=1, linewidth=0, cmap='Greens')

Ts = [T_obs, T_obs_hat, T_obs_tfce, T_obs_tfce_hat]
titles = ['T statistic', 'T with "hat"', 'TFCE statistic', 'TFCE w/"hat" stat']
for ii, (t, title) in enumerate(zip(Ts, titles)):
    ax = fig.add_subplot(2, 4, ii + 1, projection='3d')
    ax.plot_surface(x, y, t, **kwargs)
    ax.set_xticks([])
    ax.set_yticks([])
    ax.set_title(title)

p_lims = [1.3, -np.log10(1.0 / n_permutations)]
pvals = [ps, ps_hat, ps_tfce, ps_tfce_hat]
titles = ['Standard clustering', 'Clust. w/"hat"',
          'Clust. w/TFCE', 'Clust. w/TFCE+"hat"']
axs = []
for ii, (p, title) in enumerate(zip(pvals, titles)):
    ax = fig.add_subplot(2, 4, 5 + ii)
    plt.imshow(p, cmap='Purples', vmin=p_lims[0], vmax=p_lims[1])
    ax.set_xticks([])
    ax.set_yticks([])
    ax.set_title(title)
    axs.append(ax)

plt.tight_layout()
for ax in axs:
    cbar = plt.colorbar(ax=ax, shrink=0.75, orientation='horizontal',
                        fraction=0.1, pad=0.025)
    cbar.set_label('-log10(p)')
    cbar.set_ticks(p_lims)
    cbar.set_ticklabels(['%0.1f' % p for p in p_lims])

plt.show()


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal.ipynb".
The first task is:
Compute statistic
To use an algorithm optimized for spatio-temporal clustering, we
just pass the spatial connectivity matrix (instead of spatio-temporal)
Can you write Python code for it?

print('Computing connectivity.')
connectivity = spatial_tris_connectivity(grade_to_tris(5))

#    Note that X needs to be a multi-dimensional array of shape
#    samples (subjects) x time x space, so we permute dimensions
X = np.transpose(X, [2, 1, 0])

#    Now let's actually do the clustering. This can take a long time...
#    Here we set the threshold quite high to reduce computation.
p_threshold = 0.001
t_threshold = -stats.distributions.t.ppf(p_threshold / 2., n_subjects - 1)
print('Clustering.')
T_obs, clusters, cluster_p_values, H0 = clu = \
    spatio_temporal_cluster_1samp_test(X, connectivity=connectivity, n_jobs=1,
                                       threshold=t_threshold)
#    Now select the clusters that are sig. at p < 0.05 (note that this value
#    is multiple-comparisons corrected).
good_cluster_inds = np.where(cluster_p_values < 0.05)[0]


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal.ipynb".
The first task is:
It's a good idea to spatially smooth the data, and for visualization
purposes, let's morph these to fsaverage, which is a grade 5 source space
with vertices 0:10242 for each hemisphere. Usually you'd have to morph
each subject's data separately (and you might want to use morph_data
instead), but here since all estimates are on 'sample' we can use one
morph matrix for all the heavy lifting.
Can you write Python code for it?

fsave_vertices = [np.arange(10242), np.arange(10242)]
morph_mat = compute_morph_matrix('sample', 'fsaverage', sample_vertices,
                                 fsave_vertices, 20, subjects_dir)
n_vertices_fsave = morph_mat.shape[0]

#    We have to change the shape for the dot() to work properly
X = X.reshape(n_vertices_sample, n_times * n_subjects * 2)
print('Morphing data.')
X = morph_mat.dot(X)  # morph_mat is a sparse matrix
X = X.reshape(n_vertices_fsave, n_times, n_subjects, 2)


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'
subjects_dir = data_path + '/subjects'

tmin = -0.2
tmax = 0.3  # Use a lower tmax to reduce multiple comparisons

#   Setup for reading the raw data
raw = io.read_raw_fif(raw_fname)
events = mne.read_events(event_fname)


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal.ipynb".
The first task is:
Transform to source space
Can you write Python code for it?

fname_inv = data_path + '/MEG/sample/sample_audvis-meg-oct-6-meg-inv.fif'
snr = 3.0
lambda2 = 1.0 / snr ** 2
method = "dSPM"  # use dSPM method (could also be MNE or sLORETA)
inverse_operator = read_inverse_operator(fname_inv)
sample_vertices = [s['vertno'] for s in inverse_operator['src']]

#    Let's average and compute inverse, resampling to speed things up
evoked1 = epochs1.average()
evoked1.resample(50, npad='auto')
condition1 = apply_inverse(evoked1, inverse_operator, lambda2, method)
evoked2 = epochs2.average()
evoked2.resample(50, npad='auto')
condition2 = apply_inverse(evoked2, inverse_operator, lambda2, method)

#    Let's only deal with t > 0, cropping to reduce multiple comparisons
condition1.crop(0, None)
condition2.crop(0, None)
tmin = condition1.tmin
tstep = condition1.tstep


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal.ipynb".
The first task is:
Visualize the clusters
Can you write Python code for it?

print('Visualizing clusters.')

#    Now let's build a convenient representation of each cluster, where each
#    cluster becomes a "time point" in the SourceEstimate
stc_all_cluster_vis = summarize_clusters_stc(clu, tstep=tstep,
                                             vertices=fsave_vertices,
                                             subject='fsaverage')

#    Let's actually plot the first "time point" in the SourceEstimate, which
#    shows all the clusters, weighted by duration
subjects_dir = op.join(data_path, 'subjects')
# blue blobs are for condition A < condition B, red for A > B
brain = stc_all_cluster_vis.plot(hemi='both', subjects_dir=subjects_dir,
                                 time_label='Duration significant (ms)')
brain.set_data_time_index(0)
brain.show_view('lateral')
brain.save_image('clusters.png')


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal_2samp.ipynb".
The first task is:
.. _tut_stats_cluster_source_2samp:
2 samples permutation test on source data with spatio-temporal clustering
Tests if the source space data are significantly different between
2 groups of subjects (simulated here using one subject's data).
The multiple comparisons problem is addressed with a cluster-level
permutation test across space and time.
Can you write Python code for it?

# Authors: Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
#          Eric Larson <larson.eric.d@gmail.com>
# License: BSD (3-clause)

import os.path as op
import numpy as np
from scipy import stats as stats

import mne
from mne import spatial_tris_connectivity, grade_to_tris
from mne.stats import spatio_temporal_cluster_test, summarize_clusters_stc
from mne.datasets import sample

print(__doc__)


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal_2samp.ipynb".
The first task is:
Compute statistic
To use an algorithm optimized for spatio-temporal clustering, we
just pass the spatial connectivity matrix (instead of spatio-temporal)
Can you write Python code for it?

print('Computing connectivity.')
connectivity = spatial_tris_connectivity(grade_to_tris(5))

#    Note that X needs to be a list of multi-dimensional array of shape
#    samples (subjects_k) x time x space, so we permute dimensions
X1 = np.transpose(X1, [2, 1, 0])
X2 = np.transpose(X2, [2, 1, 0])
X = [X1, X2]

#    Now let's actually do the clustering. This can take a long time...
#    Here we set the threshold quite high to reduce computation.
p_threshold = 0.0001
f_threshold = stats.distributions.f.ppf(1. - p_threshold / 2.,
                                        n_subjects1 - 1, n_subjects2 - 1)
print('Clustering.')
T_obs, clusters, cluster_p_values, H0 = clu =\
    spatio_temporal_cluster_test(X, connectivity=connectivity, n_jobs=1,
                                 threshold=f_threshold)
#    Now select the clusters that are sig. at p < 0.05 (note that this value
#    is multiple-comparisons corrected).
good_cluster_inds = np.where(cluster_p_values < 0.05)[0]


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal_2samp.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

data_path = sample.data_path()
stc_fname = data_path + '/MEG/sample/sample_audvis-meg-lh.stc'
subjects_dir = data_path + '/subjects'

# Load stc to in common cortical space (fsaverage)
stc = mne.read_source_estimate(stc_fname)
stc.resample(50, npad='auto')

stc = mne.morph_data('sample', 'fsaverage', stc, grade=5, smooth=20,
                     subjects_dir=subjects_dir)
n_vertices_fsave, n_times = stc.data.shape
tstep = stc.tstep

n_subjects1, n_subjects2 = 7, 9
print('Simulating data for %d and %d subjects.' % (n_subjects1, n_subjects2))

#    Let's make sure our results replicate, so set the seed.
np.random.seed(0)
X1 = np.random.randn(n_vertices_fsave, n_times, n_subjects1) * 10
X2 = np.random.randn(n_vertices_fsave, n_times, n_subjects2) * 10
X1[:, :, :] += stc.data[:, :, np.newaxis]
# make the activity bigger for the second set of subjects
X2[:, :, :] += 3 * stc.data[:, :, np.newaxis]

#    We want to compare the overall activity levels for each subject
X1 = np.abs(X1)  # only magnitude
X2 = np.abs(X2)  # only magnitude


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal_repeated_measures_anova.ipynb".
The first task is:
It's a good idea to spatially smooth the data, and for visualization
purposes, let's morph these to fsaverage, which is a grade 5 source space
with vertices 0:10242 for each hemisphere. Usually you'd have to morph
each subject's data separately (and you might want to use morph_data
instead), but here since all estimates are on 'sample' we can use one
morph matrix for all the heavy lifting.
Can you write Python code for it?

fsave_vertices = [np.arange(10242), np.array([], int)]  # right hemi is empty
morph_mat = compute_morph_matrix('sample', 'fsaverage', sample_vertices,
                                 fsave_vertices, 20, subjects_dir)
n_vertices_fsave = morph_mat.shape[0]

#    We have to change the shape for the dot() to work properly
X = X.reshape(n_vertices_sample, n_times * n_subjects * 4)
print('Morphing data.')
X = morph_mat.dot(X)  # morph_mat is a sparse matrix
X = X.reshape(n_vertices_fsave, n_times, n_subjects, 4)


I am working on the file "0.12/_downloads/plot_stats_cluster_spatio_temporal_repeated_measures_anova.ipynb".
The first task is:
Now we need to prepare the group matrix for the ANOVA statistic. To make the
clustering function work correctly with the ANOVA function X needs to be a
list of multi-dimensional arrays (one per condition) of shape: samples
(subjects) x time x space.
First we permute dimensions, then split the array into a list of conditions
and discard the empty dimension resulting from the split using numpy squeeze.
Can you write Python code for it?

X = np.transpose(X, [2, 1, 0, 3])  #
X = [np.squeeze(x) for x in np.split(X, 4, axis=-1)]


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency.ipynb".
The first task is:
.. _tut_stats_cluster_sensor_2samp_tfr:
Non-parametric between conditions cluster statistic on single trial power
This script shows how to compare clusters in time-frequency
power estimates between conditions. It uses a non-parametric
statistical procedure based on permutations and cluster
level statistics.
The procedure consists in:

extracting epochs for 2 conditions
compute single trial power estimates
baseline line correct the power estimates (power ratios)
compute stats to see if the power estimates are significantly different
    between conditions.
Can you write Python code for it?

# Authors: Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
#
# License: BSD (3-clause)

import numpy as np
import matplotlib.pyplot as plt

import mne
from mne import io
from mne.time_frequency import single_trial_power
from mne.stats import permutation_cluster_test
from mne.datasets import sample

print(__doc__)


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency.ipynb".
The first task is:
Compute ratio with baseline power (be sure to correct time vector with
decimation factor)
Can you write Python code for it?

baseline_mask = times[::decim] < 0
epochs_baseline_1 = np.mean(epochs_power_1[:, :, baseline_mask], axis=2)
epochs_power_1 /= epochs_baseline_1[..., np.newaxis]
epochs_baseline_2 = np.mean(epochs_power_2[:, :, baseline_mask], axis=2)
epochs_power_2 /= epochs_baseline_2[..., np.newaxis]


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency.ipynb".
The first task is:
Compute statistic
Can you write Python code for it?

threshold = 6.0
T_obs, clusters, cluster_p_values, H0 = \
    permutation_cluster_test([epochs_power_1, epochs_power_2],
                             n_permutations=100, threshold=threshold, tail=0)


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency.ipynb".
The first task is:
Factor to downsample the temporal dimension of the PSD computed by
single_trial_power.  Decimation occurs after frequency decomposition and can
be used to reduce memory usage (and possibly comptuational time of downstream
operations such as nonparametric statistics) if you don't need high
spectrotemporal resolution.
Can you write Python code for it?

decim = 2
frequencies = np.arange(7, 30, 3)  # define frequencies of interest
sfreq = raw.info['sfreq']  # sampling in Hz
n_cycles = 1.5

epochs_power_1 = single_trial_power(data_condition_1, sfreq=sfreq,
                                    frequencies=frequencies,
                                    n_cycles=n_cycles, decim=decim)

epochs_power_2 = single_trial_power(data_condition_2, sfreq=sfreq,
                                    frequencies=frequencies,
                                    n_cycles=n_cycles, decim=decim)

epochs_power_1 = epochs_power_1[:, 0, :, :]  # only 1 channel to get 3D matrix
epochs_power_2 = epochs_power_2[:, 0, :, :]  # only 1 channel to get 3D matrix


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency.ipynb".
The first task is:
View time-frequency plots
Can you write Python code for it?

plt.clf()
plt.subplots_adjust(0.12, 0.08, 0.96, 0.94, 0.2, 0.43)
plt.subplot(2, 1, 1)
evoked_contrast = np.mean(data_condition_1, 0) - np.mean(data_condition_2, 0)
plt.plot(times, evoked_contrast.T)
plt.title('Contrast of evoked response (%s)' % ch_name)
plt.xlabel('time (ms)')
plt.ylabel('Magnetic Field (fT/cm)')
plt.xlim(times[0], times[-1])
plt.ylim(-100, 200)

plt.subplot(2, 1, 2)

# Create new stats image with only significant clusters
T_obs_plot = np.nan * np.ones_like(T_obs)
for c, p_val in zip(clusters, cluster_p_values):
    if p_val <= 0.05:
        T_obs_plot[c] = T_obs[c]

plt.imshow(T_obs,
           extent=[times[0], times[-1], frequencies[0], frequencies[-1]],
           aspect='auto', origin='lower', cmap='RdBu_r')
plt.imshow(T_obs_plot,
           extent=[times[0], times[-1], frequencies[0], frequencies[-1]],
           aspect='auto', origin='lower', cmap='RdBu_r')

plt.xlabel('time (ms)')
plt.ylabel('Frequency (Hz)')
plt.title('Induced power (%s)' % ch_name)
plt.show()


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency_repeated_measures_anova.ipynb".
The first task is:
.. _tut_stats_cluster_sensor_rANOVA_tfr:
Mass-univariate twoway repeated measures ANOVA on single trial power
This script shows how to conduct a mass-univariate repeated measures
ANOVA. As the model to be fitted assumes two fully crossed factors,
we will study the interplay between perceptual modality
(auditory VS visual) and the location of stimulus presentation
(left VS right). Here we use single trials as replications
(subjects) while iterating over time slices plus frequency bands
for to fit our mass-univariate model. For the sake of simplicity we
will confine this analysis to one single channel of which we know
that it exposes a strong induced response. We will then visualize
each effect by creating a corresponding mass-univariate effect
image. We conclude with accounting for multiple comparisons by
performing a permutation clustering test using the ANOVA as
clustering function. The results final will be compared to
multiple comparisons using False Discovery Rate correction.
Can you write Python code for it?

# Authors: Denis Engemann <denis.engemann@gmail.com>
#          Eric Larson <larson.eric.d@gmail.com>
#          Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
#
# License: BSD (3-clause)

import numpy as np
import matplotlib.pyplot as plt

import mne
from mne import io
from mne.time_frequency import single_trial_power
from mne.stats import f_threshold_mway_rm, f_mway_rm, fdr_correction
from mne.datasets import sample

print(__doc__)


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency_repeated_measures_anova.ipynb".
The first task is:
Account for multiple comparisons using FDR versus permutation clustering test
First we need to slightly modify the ANOVA function to be suitable for
the clustering procedure. Also want to set some defaults.
Let's first override effects to confine the analysis to the interaction
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency_repeated_measures_anova.ipynb".
The first task is:
Create TFR representations for all conditions
Can you write Python code for it?

epochs_power = list()
for condition in [epochs[k].get_data()[:, 97:98, :] for k in event_id]:
    this_power = single_trial_power(condition, sfreq=sfreq,
                                    frequencies=frequencies, n_cycles=n_cycles,
                                    decim=decim)
    this_power = this_power[:, 0, :, :]  # we only have one channel.
    # Compute ratio with baseline power (be sure to correct time vector with
    # decimation factor)
    epochs_baseline = np.mean(this_power[:, :, baseline_mask], axis=2)
    this_power /= epochs_baseline[..., np.newaxis]
    epochs_power.append(this_power)


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency_repeated_measures_anova.ipynb".
The first task is:
Create new stats image with only significant clusters
Can you write Python code for it?

good_clusers = np.where(cluster_p_values < .05)[0]
T_obs_plot = np.ma.masked_array(T_obs,
                                np.invert(clusters[np.squeeze(good_clusers)]))

plt.figure()
for f_image, cmap in zip([T_obs, T_obs_plot], [plt.cm.gray, 'RdBu_r']):
    plt.imshow(f_image, cmap=cmap, extent=[times[0], times[-1],
               frequencies[0], frequencies[-1]], aspect='auto',
               origin='lower')
plt.xlabel('time (ms)')
plt.ylabel('Frequency (Hz)')
plt.title('Time-locked response for \'modality by location\' (%s)\n'
          ' cluster-level corrected (p <= 0.05)' % ch_name)
plt.show()


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency_repeated_measures_anova.ipynb".
The first task is:
Now using FDR
Can you write Python code for it?

mask, _ = fdr_correction(pvals[2])
T_obs_plot2 = np.ma.masked_array(T_obs, np.invert(mask))

plt.figure()
for f_image, cmap in zip([T_obs, T_obs_plot2], [plt.cm.gray, 'RdBu_r']):
    plt.imshow(f_image, cmap=cmap, extent=[times[0], times[-1],
               frequencies[0], frequencies[-1]], aspect='auto',
               origin='lower')

plt.xlabel('time (ms)')
plt.ylabel('Frequency (Hz)')
plt.title('Time-locked response for \'modality by location\' (%s)\n'
          ' FDR corrected (p <= 0.05)' % ch_name)
plt.show()

# Both, cluster level and FDR correction help getting rid of
# putatively spots we saw in the naive f-images.


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency_repeated_measures_anova.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_raw-eve.fif'
event_id = 1
tmin = -0.2
tmax = 0.5

# Setup for reading the raw data
raw = io.read_raw_fif(raw_fname)
events = mne.read_events(event_fname)

include = []
raw.info['bads'] += ['MEG 2443']  # bads

# picks MEG gradiometers
picks = mne.pick_types(raw.info, meg='grad', eeg=False, eog=True,
                       stim=False, include=include, exclude='bads')

ch_name = raw.info['ch_names'][picks[0]]

# Load conditions
reject = dict(grad=4000e-13, eog=150e-6)
event_id = dict(aud_l=1, aud_r=2, vis_l=3, vis_r=4)
epochs = mne.Epochs(raw, events, event_id, tmin, tmax,
                    picks=picks, baseline=(None, 0),
                    reject=reject)


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency_repeated_measures_anova.ipynb".
The first task is:
Setup repeated measures ANOVA
We will tell the ANOVA how to interpret the data matrix in terms of factors.
This is done via the factor levels argument which is a list of the number
factor levels for each factor.
Can you write Python code for it?

n_conditions = len(epochs.event_id)
n_replications = epochs.events.shape[0] / n_conditions

factor_levels = [2, 2]  # number of levels in each factor
effects = 'A*B'  # this is the default signature for computing all effects
# Other possible options are 'A' or 'B' for the corresponding main effects
# or 'A:B' for the interaction effect only (this notation is borrowed from the
# R formula language)
n_frequencies = len(frequencies)
n_times = len(times[::decim])


I am working on the file "0.12/_downloads/plot_stats_cluster_time_frequency_repeated_measures_anova.ipynb".
The first task is:
We have to make sure all conditions have the same counts, as the ANOVA
expects a fully balanced data matrix and does not forgive imbalances that
generously (risk of type-I error).
Can you write Python code for it?

epochs.equalize_event_counts(event_id, copy=False)
# Time vector
times = 1e3 * epochs.times  # change unit to ms

# Factor to down-sample the temporal dimension of the PSD computed by
# single_trial_power.
decim = 2
frequencies = np.arange(7, 30, 3)  # define frequencies of interest
sfreq = raw.info['sfreq']  # sampling in Hz
n_cycles = frequencies / frequencies[0]
baseline_mask = times[::decim] < 0


I am working on the file "0.12/_downloads/plot_stats_spatio_temporal_cluster_sensors.ipynb".
The first task is:
Compute permutation statistic
How does it work? We use clustering to bind together features which are
similar. Our features are the magnetic fields measured over our sensor
array at different times. This reduces the multiple comparison problem.
To compute the actual test-statistic, we first sum all F-values in all
clusters. We end up with one statistic for each cluster.
Then we generate a distribution from the data by shuffling our conditions
between our samples and recomputing our clusters and the test statistics.
We test for the significance of a given cluster by computing the probability
of observing a cluster of that size. For more background read:
Maris/Oostenveld (2007), "Nonparametric statistical testing of EEG- and
MEG-data" Journal of Neuroscience Methods, Vol. 164, No. 1., pp. 177-190.
doi:10.1016/j.jneumeth.2007.03.024
Can you write Python code for it?

# set cluster threshold
threshold = 50.0  # very high, but the test is quite sensitive on this data
# set family-wise p-value
p_accept = 0.001

cluster_stats = spatio_temporal_cluster_test(X, n_permutations=1000,
                                             threshold=threshold, tail=1,
                                             n_jobs=1,
                                             connectivity=connectivity)

T_obs, clusters, p_values, _ = cluster_stats
good_cluster_inds = np.where(p_values < p_accept)[0]


I am working on the file "0.12/_downloads/plot_stats_spatio_temporal_cluster_sensors.ipynb".
The first task is:
Note. The same functions work with source estimate. The only differences
are the origin of the data, the size, and the connectivity definition.
It can be used for single trials or for groups of subjects.
Visualize clusters
Can you write Python code for it?

# configure variables for visualization
times = epochs.times * 1e3
colors = 'r', 'r', 'steelblue', 'steelblue'
linestyles = '-', '--', '-', '--'

# grand average as numpy arrray
grand_ave = np.array(X).mean(axis=1)

# get sensor positions via layout
pos = mne.find_layout(epochs.info).pos

# loop over significant clusters
for i_clu, clu_idx in enumerate(good_cluster_inds):
    # unpack cluster information, get unique indices
    time_inds, space_inds = np.squeeze(clusters[clu_idx])
    ch_inds = np.unique(space_inds)
    time_inds = np.unique(time_inds)

    # get topography for F stat
    f_map = T_obs[time_inds, ...].mean(axis=0)

    # get signals at significant sensors
    signals = grand_ave[..., ch_inds].mean(axis=-1)
    sig_times = times[time_inds]

    # create spatial mask
    mask = np.zeros((f_map.shape[0], 1), dtype=bool)
    mask[ch_inds, :] = True

    # initialize figure
    fig, ax_topo = plt.subplots(1, 1, figsize=(10, 3))
    title = 'Cluster #{0}'.format(i_clu + 1)
    fig.suptitle(title, fontsize=14)

    # plot average test statistic and mark significant sensors
    image, _ = plot_topomap(f_map, pos, mask=mask, axes=ax_topo,
                            cmap='Reds', vmin=np.min, vmax=np.max)

    # advanced matplotlib for showing image with figure and colorbar
    # in one plot
    divider = make_axes_locatable(ax_topo)

    # add axes for colorbar
    ax_colorbar = divider.append_axes('right', size='5%', pad=0.05)
    plt.colorbar(image, cax=ax_colorbar)
    ax_topo.set_xlabel('Averaged F-map ({:0.1f} - {:0.1f} ms)'.format(
        *sig_times[[0, -1]]
    ))

    # add new axis for time courses and plot time courses
    ax_signals = divider.append_axes('right', size='300%', pad=1.2)
    for signal, name, col, ls in zip(signals, condition_names, colors,
                                     linestyles):
        ax_signals.plot(times, signal, color=col, linestyle=ls, label=name)

    # add information
    ax_signals.axvline(0, color='k', linestyle=':', label='stimulus onset')
   ...


I am working on the file "0.12/_downloads/plot_stats_spatio_temporal_cluster_sensors.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'
event_id = {'Aud_L': 1, 'Aud_R': 2, 'Vis_L': 3, 'Vis_R': 4}
tmin = -0.2
tmax = 0.5

# Setup for reading the raw data
raw = mne.io.read_raw_fif(raw_fname, preload=True)
raw.filter(1, 30)
events = mne.read_events(event_fname)


I am working on the file "0.12/_downloads/plot_temporal_whitening.ipynb".
The first task is:
Plot the different time series and PSDs
Can you write Python code for it?

plt.close('all')
plt.figure()
plt.plot(d[:100], label='signal')
plt.plot(d_[:100], label='regenerated signal')
plt.legend()

plt.figure()
plt.psd(d, Fs=raw.info['sfreq'], NFFT=2048)
plt.psd(innovation, Fs=raw.info['sfreq'], NFFT=2048)
plt.psd(d_, Fs=raw.info['sfreq'], NFFT=2048, linestyle='--')
plt.legend(('Signal', 'Innovation', 'Regenerated signal'))
plt.show()


I am working on the file "0.12/_downloads/plot_tf_dics.ipynb".
The first task is:
Read raw data
Can you write Python code for it?

raw = mne.io.read_raw_fif(raw_fname, preload=True)
raw.info['bads'] = ['MEG 2443']  # 1 bad MEG channel

# Pick a selection of magnetometer channels. A subset of all channels was used
# to speed up the example. For a solution based on all MEG channels use
# meg=True, selection=None and add mag=4e-12 to the reject dictionary.
left_temporal_channels = mne.read_selection('Left-temporal')
picks = mne.pick_types(raw.info, meg='mag', eeg=False, eog=False,
                       stim=False, exclude='bads',
                       selection=left_temporal_channels)
raw.pick_channels([raw.ch_names[pick] for pick in picks])
reject = dict(mag=4e-12)
# Re-normalize our empty-room projectors, which should be fine after
# subselection
raw.info.normalize_proj()

# Setting time windows. Note that tmin and tmax are set so that time-frequency
# beamforming will be performed for a wider range of time points than will
# later be displayed on the final spectrogram. This ensures that all time bins
# displayed represent an average of an equal number of time windows.
tmin, tmax, tstep = -0.55, 0.75, 0.05  # s
tmin_plot, tmax_plot = -0.3, 0.5  # s

# Read epochs
event_id = 1
events = mne.read_events(event_fname)
epochs = mne.Epochs(raw, events, event_id, tmin, tmax,
                    baseline=None, preload=True, proj=True, reject=reject)

# Read empty room noise raw data
raw_noise = mne.io.read_raw_fif(noise_fname, preload=True)
raw_noise.info['bads'] = ['MEG 2443']  # 1 bad MEG channel
raw_noise.pick_channels([raw_noise.ch_names[pick] for pick in picks])
raw_noise.info.normalize_proj()

# Create noise epochs and make sure the number of noise epochs corresponds to
# the number of data epochs
events_noise = make_fixed_length_events(raw_noise, event_id)
epochs_noise = mne.Epochs(raw_noise, events_noise, event_id, tmin_plot,
                          tmax_plot, baseline=None, preload=True, proj=True,
                          reject=reject)
epochs_noise.info.normalize_proj()
epochs_noise.apply_proj()
# then make sure the number of...


I am working on the file "0.12/_downloads/plot_tf_dics.ipynb".
The first task is:
Time-frequency beamforming based on DICS
Can you write Python code for it?

# Setting frequency bins as in Dalal et al. 2008
freq_bins = [(4, 12), (12, 30), (30, 55), (65, 300)]  # Hz
win_lengths = [0.3, 0.2, 0.15, 0.1]  # s
# Then set FFTs length for each frequency range.
# Should be a power of 2 to be faster.
n_ffts = [256, 128, 128, 128]

# Subtract evoked response prior to computation?
subtract_evoked = False

# Calculating noise cross-spectral density from empty room noise for each
# frequency bin and the corresponding time window length. To calculate noise
# from the baseline period in the data, change epochs_noise to epochs
noise_csds = []
for freq_bin, win_length, n_fft in zip(freq_bins, win_lengths, n_ffts):
    noise_csd = compute_epochs_csd(epochs_noise, mode='fourier',
                                   fmin=freq_bin[0], fmax=freq_bin[1],
                                   fsum=True, tmin=-win_length, tmax=0,
                                   n_fft=n_fft)
    noise_csds.append(noise_csd)

# Computing DICS solutions for time-frequency windows in a label in source
# space for faster computation, use label=None for full solution
stcs = tf_dics(epochs, forward, noise_csds, tmin, tmax, tstep, win_lengths,
               freq_bins=freq_bins, subtract_evoked=subtract_evoked,
               n_ffts=n_ffts, reg=0.001, label=label)

# Plotting source spectrogram for source with maximum activity
# Note that tmin and tmax are set to display a time range that is smaller than
# the one for which beamforming estimates were calculated. This ensures that
# all time bins shown are a result of smoothing across an identical number of
# time windows.
plot_source_spectrogram(stcs, freq_bins, tmin=tmin_plot, tmax=tmax_plot,
                        source_index=None, colorbar=True)


I am working on the file "0.12/_downloads/plot_tf_lcmv.ipynb".
The first task is:
Time-frequency beamforming based on LCMV
Can you write Python code for it?

# Setting frequency bins as in Dalal et al. 2008 (high gamma was subdivided)
freq_bins = [(4, 12), (12, 30), (30, 55), (65, 299)]  # Hz
win_lengths = [0.3, 0.2, 0.15, 0.1]  # s

# Setting the time step
tstep = 0.05

# Setting the whitened data covariance regularization parameter
data_reg = 0.001

# Subtract evoked response prior to computation?
subtract_evoked = False

# Calculating covariance from empty room noise. To use baseline data as noise
# substitute raw for raw_noise, epochs.events for epochs_noise.events, tmin for
# desired baseline length, and 0 for tmax_plot.
# Note, if using baseline data, the averaged evoked response in the baseline
# period should be flat.
noise_covs = []
for (l_freq, h_freq) in freq_bins:
    raw_band = raw_noise.copy()
    raw_band.filter(l_freq, h_freq, method='iir', n_jobs=1)
    epochs_band = mne.Epochs(raw_band, epochs_noise.events, event_id,
                             tmin=tmin_plot, tmax=tmax_plot, baseline=None,
                             proj=True)

    noise_cov = compute_covariance(epochs_band, method='shrunk')
    noise_covs.append(noise_cov)
    del raw_band  # to save memory

# Computing LCMV solutions for time-frequency windows in a label in source
# space for faster computation, use label=None for full solution
stcs = tf_lcmv(epochs, forward, noise_covs, tmin, tmax, tstep, win_lengths,
               freq_bins=freq_bins, subtract_evoked=subtract_evoked,
               reg=data_reg, label=label)

# Plotting source spectrogram for source with maximum activity.
# Note that tmin and tmax are set to display a time range that is smaller than
# the one for which beamforming estimates were calculated. This ensures that
# all time bins shown are a result of smoothing across an identical number of
# time windows.
plot_source_spectrogram(stcs, freq_bins, tmin=tmin_plot, tmax=tmax_plot,
                        source_index=None, colorbar=True)


I am working on the file "0.12/_downloads/plot_time_frequency_mixed_norm_inverse.ipynb".
The first task is:
Run solver
Can you write Python code for it?

# alpha_space regularization parameter is between 0 and 100 (100 is high)
alpha_space = 50.  # spatial regularization parameter
# alpha_time parameter promotes temporal smoothness
# (0 means no temporal regularization)
alpha_time = 1.  # temporal regularization parameter

loose, depth = 0.2, 0.9  # loose orientation & depth weighting

# Compute dSPM solution to be used as weights in MxNE
inverse_operator = make_inverse_operator(evoked.info, forward, cov,
                                         loose=loose, depth=depth)
stc_dspm = apply_inverse(evoked, inverse_operator, lambda2=1. / 9.,
                         method='dSPM')

# Compute TF-MxNE inverse solution
stc, residual = tf_mixed_norm(evoked, forward, cov, alpha_space, alpha_time,
                              loose=loose, depth=depth, maxit=200, tol=1e-4,
                              weights=stc_dspm, weights_min=8., debias=True,
                              wsize=16, tstep=4, window=0.05,
                              return_residual=True)

# Crop to remove edges
stc.crop(tmin=-0.05, tmax=0.3)
evoked.crop(tmin=-0.05, tmax=0.3)
residual.crop(tmin=-0.05, tmax=0.3)

# Show the evoked response and the residual for gradiometers
ylim = dict(grad=[-120, 120])
evoked.pick_types(meg='grad', exclude='bads')
evoked.plot(titles=dict(grad='Evoked Response: Gradiometers'), ylim=ylim,
            proj=True)

residual.pick_types(meg='grad', exclude='bads')
residual.plot(titles=dict(grad='Residuals: Gradiometers'), ylim=ylim,
              proj=True)


I am working on the file "0.12/_downloads/plot_time_frequency_mixed_norm_inverse.ipynb".
The first task is:
View in 2D and 3D ("glass" brain like 3D plot)
Can you write Python code for it?

plot_sparse_source_estimates(forward['src'], stc, bgcolor=(1, 1, 1),
                             opacity=0.1, fig_name="TF-MxNE (cond %s)"
                             % condition, modes=['sphere'], scale_factors=[1.])

time_label = 'TF-MxNE time=%0.2f ms'
clim = dict(kind='value', lims=[10e-9, 15e-9, 20e-9])
brain = stc.plot('sample', 'inflated', 'rh', clim=clim, time_label=time_label,
                 smoothing_steps=5, subjects_dir=subjects_dir)
brain.show_view('medial')
brain.set_data_time_index(120)
brain.add_label("V1", color="yellow", scalar_thresh=.5, borders=True)
brain.add_label("V2", color="red", scalar_thresh=.5, borders=True)


I am working on the file "0.12/_downloads/plot_time_frequency_simulated.ipynb".
The first task is:
========================================================
Time-frequency on simulated data (Multitaper vs. Morlet)
========================================================
This examples demonstrates on simulated data the different time-frequency
estimation methods. It shows the time-frequency resolution trade-off
and the problem of estimation variance.
Can you write Python code for it?

# Authors: Hari Bharadwaj <hari@nmr.mgh.harvard.edu>
#          Denis Engemann <denis.engemann@gmail.com>
#
# License: BSD (3-clause)

import numpy as np

from mne import create_info, EpochsArray
from mne.time_frequency import tfr_multitaper, tfr_stockwell, tfr_morlet

print(__doc__)


I am working on the file "0.12/_downloads/plot_time_frequency_simulated.ipynb".
The first task is:
Consider different parameter possibilities for multitaper convolution
Can you write Python code for it?

freqs = np.arange(5., 100., 3.)

# You can trade time resolution or frequency resolution or both
# in order to get a reduction in variance

# (1) Least smoothing (most variance/background fluctuations).
n_cycles = freqs / 2.
time_bandwidth = 2.0  # Least possible frequency-smoothing (1 taper)
power = tfr_multitaper(epochs, freqs=freqs, n_cycles=n_cycles,
                       time_bandwidth=time_bandwidth, return_itc=False)
# Plot results. Baseline correct based on first 100 ms.
power.plot([0], baseline=(0., 0.1), mode='mean', vmin=-1., vmax=3.,
           title='Sim: Least smoothing, most variance')

# (2) Less frequency smoothing, more time smoothing.
n_cycles = freqs  # Increase time-window length to 1 second.
time_bandwidth = 4.0  # Same frequency-smoothing as (1) 3 tapers.
power = tfr_multitaper(epochs, freqs=freqs, n_cycles=n_cycles,
                       time_bandwidth=time_bandwidth, return_itc=False)
# Plot results. Baseline correct based on first 100 ms.
power.plot([0], baseline=(0., 0.1), mode='mean', vmin=-1., vmax=3.,
           title='Sim: Less frequency smoothing, more time smoothing')

# (3) Less time smoothing, more frequency smoothing.
n_cycles = freqs / 2.
time_bandwidth = 8.0  # Same time-smoothing as (1), 7 tapers.
power = tfr_multitaper(epochs, freqs=freqs, n_cycles=n_cycles,
                       time_bandwidth=time_bandwidth, return_itc=False)
# Plot results. Baseline correct based on first 100 ms.
power.plot([0], baseline=(0., 0.1), mode='mean', vmin=-1., vmax=3.,
           title='Sim: Less time smoothing, more frequency smoothing')

# #############################################################################
# Stockwell (S) transform

# S uses a Gaussian window to balance temporal and spectral resolution
# Importantly, frequency bands are phase-normalized, hence strictly comparable
# with regard to timing, and, the input signal can be recoverd from the
# transform in a lossless way if we disregard numerical errors.

fmin, fmax = freqs[[0, -1]]
for width in (0.7, 3.0):
   ...


I am working on the file "0.12/_downloads/plot_time_frequency_simulated.ipynb".
The first task is:
Simulate data
Can you write Python code for it?

sfreq = 1000.0
ch_names = ['SIM0001', 'SIM0002']
ch_types = ['grad', 'grad']
info = create_info(ch_names=ch_names, sfreq=sfreq, ch_types=ch_types)

n_times = int(sfreq)  # 1 second long epochs
n_epochs = 40
seed = 42
rng = np.random.RandomState(seed)
noise = rng.randn(n_epochs, len(ch_names), n_times)

# Add a 50 Hz sinusoidal burst to the noise and ramp it.
t = np.arange(n_times, dtype=np.float) / sfreq
signal = np.sin(np.pi * 2. * 50. * t)  # 50 Hz sinusoid signal
signal[np.logical_or(t < 0.45, t > 0.55)] = 0.  # Hard windowing
on_time = np.logical_and(t >= 0.45, t <= 0.55)
signal[on_time] *= np.hanning(on_time.sum())  # Ramping
data = noise + signal

reject = dict(grad=4000)
events = np.empty((n_epochs, 3), dtype=int)
first_event_sample = 100
event_id = dict(sin50hz=1)
for k in range(n_epochs):
    events[k, :] = first_event_sample + k * n_times, 0, event_id['sin50hz']

epochs = EpochsArray(data=data, info=info, events=events, event_id=event_id,
                     reject=reject)


I am working on the file "0.12/_downloads/plot_topo_compare_conditions.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'
event_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw-eve.fif'
event_id = 1
tmin = -0.2
tmax = 0.5

#   Setup for reading the raw data
raw = mne.io.read_raw_fif(raw_fname)
events = mne.read_events(event_fname)

#   Set up pick list: MEG + STI 014 - bad channels (modify to your needs)
include = []  # or stim channels ['STI 014']
# bad channels in raw.info['bads'] will be automatically excluded

#   Set up amplitude-peak rejection values for MEG channels
reject = dict(grad=4000e-13, mag=4e-12)

# pick MEG channels
picks = mne.pick_types(raw.info, meg=True, eeg=False, stim=False, eog=True,
                       include=include, exclude='bads')

# Create epochs including different events
event_id = {'audio/left': 1, 'audio/right': 2,
            'visual/left': 3, 'visual/right': 4}
epochs = mne.Epochs(raw, events, event_id, tmin, tmax,
                    picks=picks, baseline=(None, 0), reject=reject)

# Generate list of evoked objects from conditions names
evokeds = [epochs[name].average() for name in ('left', 'right')]


I am working on the file "0.12/_downloads/plot_visualize_epochs.ipynb".
The first task is:
.. _tut_viz_epochs:
Visualize Epochs data
Can you write Python code for it?

import os.path as op

import mne

data_path = op.join(mne.datasets.sample.data_path(), 'MEG', 'sample')
raw = mne.io.read_raw_fif(op.join(data_path, 'sample_audvis_raw.fif'))
events = mne.read_events(op.join(data_path, 'sample_audvis_raw-eve.fif'))
picks = mne.pick_types(raw.info, meg='grad')
epochs = mne.Epochs(raw, events, [1, 2], picks=picks)


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
.. _tut_viz_evoked:
Visualize Evoked data
Can you write Python code for it?

import os.path as op
import numpy as np
import matplotlib.pyplot as plt

import mne


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
All plotting functions of MNE-python returns a handle to the figure instance.
When we have the handle, we can customise the plots to our liking. We can get
rid of the empty space with a simple function call.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
Finally we plot the sensor data as a topographical view. In the simple case
we plot only left auditory responses, and then we plot them all in the same
figure for comparison. Click on the individual plots to open them bigger.
Can you write Python code for it?

title = 'MNE sample data (condition : %s)'
evoked_l_aud.plot_topo(title=title % evoked_l_aud.comment)
colors = 'yellow', 'green', 'red', 'blue'
mne.viz.plot_evoked_topo(evoked, color=colors,
                         title=title % 'Left/Right Auditory/Visual')


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
In this tutorial we focus on plotting functions of :class:mne.Evoked.
Here we read the evoked object from a file. Check out
:ref:tut_epoching_and_averaging to get to this stage from raw data.
Can you write Python code for it?

data_path = mne.datasets.sample.data_path()
fname = op.join(data_path, 'MEG', 'sample', 'sample_audvis-ave.fif')
evoked = mne.read_evokeds(fname, baseline=(None, 0), proj=True)
print(evoked)


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
Let's start with a simple one. We plot event related potentials / fields
(ERP/ERF). The bad channels are not plotted by default. Here we explicitly
set the exclude parameter to show the bad channels in red. All plotting
functions of MNE-python return a handle to the figure instance. When we have
the handle, we can customise the plots to our liking.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
Notice the legend on the left. The colors would suggest that there may be two
separate sources for the signals. This wasn't obvious from the first figure.
Try painting the slopes with left mouse button. It should open a new window
with topomaps (scalp plots) of the average over the painted area. There is
also a function for drawing topomaps separately.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
Or we can automatically select the peaks.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
Visualizing field lines in 3D
We now compute the field maps to project MEG and EEG data to MEG helmet
and scalp surface.
To do this we'll need coregistration information. See
:ref:tut_forward for more details.
Here we just illustrate usage.
Can you write Python code for it?

subjects_dir = data_path + '/subjects'
trans_fname = data_path + '/MEG/sample/sample_audvis_raw-trans.fif'

maps = mne.make_field_map(evoked_l_aud, trans=trans_fname, subject='sample',
                          subjects_dir=subjects_dir, n_jobs=1)

# explore several points in time
field_map = evoked_l_aud.plot_field(maps, time=.1)


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
We can also plot the activations as images. The time runs along the x-axis
and the channels along the y-axis. The amplitudes are color coded so that
the amplitudes from negative to positive translates to shift from blue to
red. White means zero amplitude. You can use the cmap parameter to define
the color map yourself. The accepted values include all matplotlib colormaps.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_visualize_evoked.ipynb".
The first task is:
You can take a look at the documentation of :func:mne.Evoked.plot_topomap
or simply write evoked_r_aud.plot_topomap? in your python console to
see the different parameters you can pass to this function. Most of the
plotting functions also accept axes parameter. With that, you can
customise your plots even further. First we shall create a set of matplotlib
axes in a single figure and plot all of our evoked categories next to each
other.
Can you write Python code for it?

fig, ax = plt.subplots(1, 5)
evoked_l_aud.plot_topomap(times=0.1, axes=ax[0], show=False)
evoked_r_aud.plot_topomap(times=0.1, axes=ax[1], show=False)
evoked_l_vis.plot_topomap(times=0.1, axes=ax[2], show=False)
evoked_r_vis.plot_topomap(times=0.1, axes=ax[3], show=True)


I am working on the file "0.12/_downloads/plot_visualize_raw.ipynb".
The first task is:
Now click the proj button at the lower right corner of the browser
window. A selection dialog should appear, where you can toggle the projectors
on and off. Notice that the first four are already applied to the data and
toggling them does not change the data. However the newly added projectors
modify the data to get rid of the EOG artifacts. Note that toggling the
projectors here doesn't actually modify the data. This is purely for visually
inspecting the effect. See :func:mne.io.Raw.del_proj to actually remove the
projectors.
Raw container also lets us easily plot the power spectra over the raw data.
See the API documentation for more info.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_visualize_raw.ipynb".
The first task is:
Now let's add some ssp projectors to the raw data. Here we read them from a
file and plot them.
Can you write Python code for it?

projs = mne.read_proj(op.join(data_path, 'sample_audvis_eog-proj.fif'))
raw.add_proj(projs)
raw.plot_projs_topomap()


I am working on the file "0.12/_downloads/plot_visualize_raw.ipynb".
The first task is:
Plotting channel wise power spectra is just as easy. The layout is inferred
from the data by default when plotting topo plots. This works for most data,
but it is also possible to define the layouts by hand. Here we select a
layout with only magnetometer channels and plot it. Then we plot the channel
wise spectra of first 30 seconds of the data.
Can you write Python code for it?

layout = mne.channels.read_layout('Vectorview-mag')
layout.plot()
raw.plot_psd_topo(tmax=30., fmin=5., fmax=60., n_fft=1024, layout=layout)


I am working on the file "0.12/_downloads/plot_visualize_raw.ipynb".
The first task is:
The channels are color coded by channel type. Generally MEG channels are
colored in different shades of blue, whereas EEG channels are black. The
channels are also sorted by channel type by default. If you want to use a
custom order for the channels, you can use order parameter of
:func:raw.plot. The scrollbar on right side of the browser window also
tells us that two of the channels are marked as bad. Bad channels are
color coded gray. By clicking the lines or channel names on the left, you can
mark or unmark a bad channel interactively. You can use +/- keys to adjust
the scale (also = works for magnifying the data). Note that the initial
scaling factors can be set with parameter scalings. If you don't know the
scaling factor for channels, you can automatically set them by passing
scalings='auto'. With pageup/pagedown and home/end keys you can
adjust the amount of data viewed at once. To see all the interactive
features, hit ? or click help in the lower left corner of the
browser window.
We read the events from a file and passed it as a parameter when calling the
method. The events are plotted as vertical lines so you can see how they
align with the raw data.
We can check where the channels reside with plot_sensors. Notice that
this method (along with many other MNE plotting functions) is callable using
any MNE data container where the channel information is available.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_visualize_raw.ipynb".
The first task is:
The first three projectors that we see are the SSP vectors from empty room
measurements to compensate for the noise. The fourth one is the average EEG
reference. These are already applied to the data and can no longer be
removed. The next six are the EOG projections that we added. Every data
channel type has two projection vectors each. Let's try the raw browser
again.
Can you write Python code for it?


I am working on the file "0.12/_downloads/plot_xdawn_denoising.ipynb".
The first task is:
XDAWN Denoising
XDAWN filters are trained from epochs, signal is projected in the sources
space and then projected back in the sensor space using only the first two
XDAWN components. The process is similar to an ICA, but is
supervised in order to maximize the signal to signal + noise ratio of the
evoked response.
WARNING: As this denoising method exploits the known events to
maximize SNR of the contrast between conditions it can lead to overfit.
To avoid a statistical analysis problem you should split epochs used
in fit with the ones used in apply method.
References
[1] Rivet, B., Souloumiac, A., Attina, V., & Gibert, G. (2009). xDAWN
algorithm to enhance evoked potentials: application to brain-computer
interface. Biomedical Engineering, IEEE Transactions on, 56(8), 2035-2043.
[2] Rivet, B., Cecotti, H., Souloumiac, A., Maby, E., & Mattout, J. (2011,
August). Theoretical analysis of xDAWN algorithm: application to an
efficient sensor selection in a P300 BCI. In Signal Processing Conference,
2011 19th European (pp. 1382-1386). IEEE.
Can you write Python code for it?

# Authors: Alexandre Barachant <alexandre.barachant@gmail.com>
#
# License: BSD (3-clause)

from mne import (io, compute_raw_covariance, read_events, pick_types,
                 Epochs)
from mne.datasets import sample
from mne.preprocessing import Xdawn
from mne.viz import plot_epochs_image

print(__doc__)

data_path = sample.data_path()


I am working on the file "0.13/_downloads/plot_artifacts_correction_filtering.ipynb".
The first task is:
Downsampling and decimation
When performing experiments where timing is critical, a signal with a high
sampling rate is desired. However, having a signal with a much higher
sampling rate than necessary needlessly consumes memory and slows down
computations operating on the data. To avoid that, you can downsample
your time series. Since downsampling raw data reduces the timing precision
of events, it is recommended only for use in procedures that do not require
optimal precision, e.g. computing EOG or ECG projectors on long recordings.

Note

A *downsampling* operation performs a low-pass (to prevent
          aliasing) followed by *decimation*, which selects every
          $N^{th}$ sample from the signal. See
          :func:`scipy.signal.resample` and
          :func:`scipy.signal.resample_poly` for examples.

Data resampling can be done with resample methods.
Can you write Python code for it?

raw.resample(100, npad="auto")  # set sampling frequency to 100Hz
raw.plot_psd(area_mode='range', tmax=10.0, picks=picks)


I am working on the file "0.13/_downloads/plot_artifacts_correction_filtering.ipynb".
The first task is:
High-pass filtering to remove slow drifts
To remove slow drifts, you can high pass.

Warning

There can be issues using high-passes greater than 0.1 Hz
             (see examples in `tut_filtering_hp_problems`),
             so apply high-pass filters with caution.

Can you write Python code for it?

raw.filter(1., None, l_trans_bandwidth='auto', filter_length='auto',
           phase='zero')
raw.plot_psd(area_mode='range', tmax=10.0, picks=picks)


I am working on the file "0.13/_downloads/plot_artifacts_correction_filtering.ipynb".
The first task is:
Removing power-line noise with low-pass filtering
If you're only interested in low frequencies, below the peaks of power-line
noise you can simply low pass filter the data.
Can you write Python code for it?

# low pass filtering below 50 Hz
raw.filter(None, 50., h_trans_bandwidth='auto', filter_length='auto',
           phase='zero')
raw.plot_psd(area_mode='range', tmax=10.0, picks=picks)


I am working on the file "0.13/_downloads/plot_artifacts_correction_filtering.ipynb".
The first task is:
Removing power-line noise with notch filtering
Removing power-line noise can be done with a Notch filter, directly on the
Raw object, specifying an array of frequency to be cut off:
Can you write Python code for it?

raw.notch_filter(np.arange(60, 241, 60), picks=picks, filter_length='auto',
                 phase='zero')
raw.plot_psd(area_mode='range', tmax=10.0, picks=picks)


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Artifact Correction with ICA
ICA finds directions in the feature space
corresponding to projections with high non-Gaussianity. We thus obtain
a decomposition into independent components, and the artifact's contribution
is localized in only a small number of components.
These components have to be correctly identified and removed.
If EOG or ECG recordings are available, they can be used in ICA to
automatically select the corresponding artifact components from the
decomposition. To do so, you have to first build an Epoch object around
blink or heartbeat event.
Can you write Python code for it?

import numpy as np

import mne
from mne.datasets import sample

from mne.preprocessing import ICA
from mne.preprocessing import create_eog_epochs, create_ecg_epochs

# getting some data ready
data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'

raw = mne.io.read_raw_fif(raw_fname, preload=True, add_eeg_ref=False)
raw.filter(1, 40, n_jobs=2)  # 1Hz high pass is often helpful for fitting ICA

picks_meg = mne.pick_types(raw.info, meg=True, eeg=False, eog=False,
                           stim=False, exclude='bads')


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Before applying artifact correction please learn about your actual artifacts
by reading tut_artifacts_detect.
Fit ICA
ICA parameters:
Can you write Python code for it?

n_components = 25  # if float, select n_components by explained variance of PCA
method = 'fastica'  # for comparison with EEGLAB try "extended-infomax" here
decim = 3  # we need sufficient statistics, not all time points -> saves time

# we will also set state of the random number generator - ICA is a
# non-deterministic algorithm, but we want to have the same decomposition
# and the same order of components each time this tutorial is run
random_state = 23


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Define the ICA object instance
Can you write Python code for it?

ica = ICA(n_components=n_components, method=method, random_state=random_state)
print(ica)


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Exercise: find and remove ECG artifacts using ICA!
Can you write Python code for it?

ecg_epochs = create_ecg_epochs(raw, tmin=-.5, tmax=.5)
ecg_inds, scores = ica.find_bads_ecg(ecg_epochs, method='ctps')
ica.plot_properties(ecg_epochs, picks=ecg_inds, psd_args={'fmax': 35.})


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Indeed it looks like an EOG, also in the average time course.
We construct a list where our reference run is the first element. Then we
can detect similar components from the other runs using
:func:mne.preprocessing.corrmap. So our template must be a tuple like
(reference_run_index, component_index):
Can you write Python code for it?

icas = [reference_ica] + icas_from_other_data
template = (0, eog_inds[0])


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Investigate our reference ICA:
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Now we can do the corrmap.
Can you write Python code for it?

fig_template, fig_detected = corrmap(icas, template=template, label="blinks",
                                     show=True, threshold=.8, ch_type='mag')


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Plot ICA components
Can you write Python code for it?

ica.plot_components()  # can you spot some potential bad guys?


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Remember, don't do this at home! Start by reading in a collection of ICA
solutions instead. Something like:
icas = [mne.preprocessing.read_ica(fname) for fname in ica_fnames]
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
That component is showing a prototypical average vertical EOG time course.
Pay attention to the labels, a customized read-out of the
:attr:ica.labels_ <mne.preprocessing.ICA.labels_>
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
We can take a look at the properties of that component, now using the
data epoched with respect to EOG events.
We will also use a little bit of smoothing along the trials axis in the
epochs image:
Can you write Python code for it?

ica.plot_properties(eog_epochs, picks=eog_inds, psd_args={'fmax': 35.},
                    image_args={'sigma': 1.})


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
What if we don't have an EOG channel?
We could either:

make a bipolar reference from frontal EEG sensors and use as virtual EOG
   channel. This can be tricky though as you can only hope that the frontal
   EEG channels only reflect EOG and not brain dynamics in the prefrontal
   cortex.
go for a semi-automated approach, using template matching.

In MNE-Python option 2 is easily achievable and it might give better results,
so let's have a look at it.
Can you write Python code for it?

from mne.preprocessing.ica import corrmap  # noqa


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
Which one is the bad EOG component?
Here we rely on our previous detection algorithm. You would need to decide
yourself if no automatic detection was available.
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
we can also take a look at multiple different components at once:
Can you write Python code for it?

ica.plot_properties(raw, picks=[1, 2], psd_args={'fmax': 35.})


I am working on the file "0.13/_downloads/plot_artifacts_correction_ica.ipynb".
The first task is:
we can see that the data were filtered so the spectrum plot is not
very informative, let's change that:
Can you write Python code for it?

ica.plot_properties(raw, picks=0, psd_args={'fmax': 35.})


I am working on the file "0.13/_downloads/plot_artifacts_correction_maxwell_filtering.ipynb".
The first task is:
Preprocess with Maxwell filtering
Can you write Python code for it?

raw = mne.io.read_raw_fif(raw_fname, add_eeg_ref=False)
raw.info['bads'] = ['MEG 2443', 'EEG 053', 'MEG 1032', 'MEG 2313']  # set bads
# Here we don't use tSSS (set st_duration) because MGH data is very clean
raw_sss = maxwell_filter(raw, cross_talk=ctc_fname, calibration=fine_cal_fname)


I am working on the file "0.13/_downloads/plot_artifacts_correction_maxwell_filtering.ipynb".
The first task is:
Set parameters
Can you write Python code for it?

raw_fname = data_path + '/MEG/sample/sample_audvis_raw.fif'
ctc_fname = data_path + '/SSS/ct_sparse_mgh.fif'
fine_cal_fname = data_path + '/SSS/sss_cal_mgh.dat'


I am working on the file "0.13/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:

Note

The rejection values can be highly data dependent. You should be careful
   when adjusting these values. Make sure not too many epochs are rejected
   and look into the cause of the rejections. Maybe it's just a matter
   of marking a single channel as bad and you'll be able to save a lot
   of data.

We then construct the epochs
Can you write Python code for it?

events = mne.find_events(raw, stim_channel='STI 014')
event_id = {"auditory/left": 1}
tmin = -0.2  # start of each epoch (200ms before the trigger)
tmax = 0.5  # end of each epoch (500ms after the trigger)
baseline = (None, 0)  # means from the first instant to t = 0
picks_meg = mne.pick_types(raw.info, meg=True, eeg=False, eog=True,
                           stim=False, exclude='bads')
epochs = mne.Epochs(raw, events, event_id, tmin, tmax, proj=True,
                    picks=picks_meg, baseline=baseline, reject=reject,
                    reject_by_annotation=True, add_eeg_ref=False)


I am working on the file "0.13/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
And plot the so-called drop log that details the reason for which some
epochs have been dropped.
Can you write Python code for it?

print(epochs.drop_log[40:45])  # only a subset
epochs.plot_drop_log()


I am working on the file "0.13/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
As the data is epoched, all the epochs overlapping with segments whose
description starts with 'bad' are rejected by default. To turn rejection off,
use keyword argument reject_by_annotation=False when constructing
:class:mne.Epochs. When working with neuromag data, the first_samp
offset of raw acquisition is also taken into account the same way as with
event lists. For more see :class:mne.Epochs and :class:mne.Annotations.
Rejecting bad epochs
When working with segmented data (Epochs) MNE offers a quite simple approach
to automatically reject/ignore bad epochs. This is done by defining
thresholds for peak-to-peak amplitude and flat signal detection.
In the following code we build Epochs from Raw object. One of the provided
parameter is named reject. It is a dictionary where every key is a
channel type as a sring and the corresponding values are peak-to-peak
rejection parameters (amplitude ranges as floats). Below we define
the peak-to-peak rejection values for gradiometers,
magnetometers and EOG:
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
Let's now interpolate the bad channels (displayed in red above)
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
Marking bad channels
Sometimes some MEG or EEG channels are not functioning properly
for various reasons. These channels should be excluded from
analysis by marking them bad as. This is done by setting the 'bads'
in the measurement info of a data container object (e.g. Raw, Epochs,
Evoked). The info['bads'] value is a Python string. Here is
example:
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
We then drop/reject the bad epochs
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_artifacts_correction_rejection.ipynb".
The first task is:
Why setting a channel bad?: If a channel does not show
a signal at all (flat) it is important to exclude it from the
analysis. If a channel as a noise level significantly higher than the
other channels it should be marked as bad. Presence of bad channels
can have terribe consequences on down stream analysis. For a flat channel
some noise estimate will be unrealistically low and
thus the current estimate calculations will give a strong weight
to the zero signal on the flat channels and will essentially vanish.
Noisy channels can also affect others when signal-space projections
or EEG average electrode reference is employed. Noisy bad channels can
also adversely affect averaging and noise-covariance matrix estimation by
causing unnecessary rejections of epochs.
Recommended ways to identify bad channels are:

Observe the quality of data during data
  acquisition and make notes of observed malfunctioning channels to
  your measurement protocol sheet.

View the on-line averages and check the condition of the channels.

Compute preliminary off-line averages with artifact rejection,
  SSP/ICA, and EEG average electrode reference computation
  off and check the condition of the channels.

View raw data with :func:mne.io.Raw.plot without SSP/ICA
  enabled and identify bad channels.

Note

Setting the bad channels should be done as early as possible in the
    analysis pipeline. That's why it's recommended to set bad channels
    the raw objects/files. If present in the raw data
    files, the bad channel selections will be automatically transferred
    to averaged files, noise-covariance matrices, forward solution
    files, and inverse operator decompositions.

The actual removal happens using :func:pick_types <mne.pick_types> with
exclude='bads' option (see picking_channels).
Instead of removing the bad channels, you can also try to repair them.
This is done by interpolation of the data from other channels.
To illustrate how to use channel interpolation...
Can you write Python code for it?

# Reading data with a bad channel marked as bad:
fname = data_path + '/MEG/sample/sample_audvis-ave.fif'
evoked = mne.read_evokeds(fname, condition='Left Auditory',
                          baseline=(None, 0))

# restrict the evoked to EEG and MEG channels
evoked.pick_types(meg=True, eeg=True, exclude=[])

# plot with bads
evoked.plot(exclude=[])

print(evoked.info['bads'])


I am working on the file "0.13/_downloads/plot_artifacts_correction_ssp.ipynb".
The first task is:
Apply SSP projections
MNE is handling projections at the level of the info,
so to register them populate the list that you find in the 'proj' field
Can you write Python code for it?

raw.info['projs'] += eog_projs + ecg_projs


I am working on the file "0.13/_downloads/plot_artifacts_correction_ssp.ipynb".
The first task is:
Artifact Correction with SSP
Can you write Python code for it?

import numpy as np

import mne
from mne.datasets import sample
from mne.preprocessing import compute_proj_ecg, compute_proj_eog

# getting some data ready
data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_filt-0-40_raw.fif'

raw = mne.io.read_raw_fif(raw_fname, preload=True, add_eeg_ref=False)
raw.set_eeg_reference()
raw.pick_types(meg=True, ecg=True, eog=True, stim=True)


I am working on the file "0.13/_downloads/plot_artifacts_correction_ssp.ipynb".
The first task is:
Looks cool right? It is however often not clear how many components you
should take and unfortunately this can have bad consequences as can be seen
interactively using the delayed SSP mode:
Can you write Python code for it?

evoked = mne.Epochs(raw, events, event_id, tmin=-0.2, tmax=0.5,
                    proj='delayed', baseline=(None, 0),
                    reject=reject).average()

# set time instants in seconds (from 50 to 150ms in a step of 10ms)
times = np.arange(0.05, 0.15, 0.01)

evoked.plot_topomap(times, proj='interactive')


I am working on the file "0.13/_downloads/plot_artifacts_correction_ssp.ipynb".
The first task is:
Yes this was it. Now MNE will apply the projs on demand at any later stage,
so watch out for proj parmeters in functions or to it explicitly
with the .apply_proj method
Demonstrate SSP cleaning on some evoked data
Can you write Python code for it?

events = mne.find_events(raw, stim_channel='STI 014')
reject = dict(grad=4000e-13, mag=4e-12, eog=150e-6)
# this can be highly data dependent
event_id = {'auditory/left': 1}

epochs_no_proj = mne.Epochs(raw, events, event_id, tmin=-0.2, tmax=0.5,
                            proj=False, baseline=(None, 0), reject=reject)
epochs_no_proj.average().plot(spatial_colors=True)

epochs_proj = mne.Epochs(raw, events, event_id, tmin=-0.2, tmax=0.5, proj=True,
                         baseline=(None, 0), reject=reject)
epochs_proj.average().plot(spatial_colors=True)


I am working on the file "0.13/_downloads/plot_artifacts_detection.ipynb".
The first task is:
Introduction to artifacts and artifact detection
Since MNE supports the data of many different acquisition systems, the
particular artifacts in your data might behave very differently from the
artifacts you can observe in our tutorials and examples.
Therefore you should be aware of the different approaches and of
the variability of artifact rejection (automatic/manual) procedures described
onwards. At the end consider always to visually inspect your data
after artifact rejection or correction.
Background: what is an artifact?
Artifacts are signal interference that can be
endogenous (biological) and exogenous (environmental).
Typical biological artifacts are head movements, eye blinks
or eye movements, heart beats. The most common environmental
artifact is due to the power line, the so-called line noise.
How to handle artifacts?
MNE deals with artifacts by first identifying them, and subsequently removing
them. Detection of artifacts can be done visually, or using automatic routines
(or a combination of both). After you know what the artifacts are, you need
remove them. This can be done by:
- *ignoring* the piece of corrupted data
- *fixing* the corrupted data

For the artifact detection the functions MNE provides depend on whether
your data is continuous (Raw) or epoch-based (Epochs) and depending on
whether your data is stored on disk or already in memory.
Detecting the artifacts without reading the complete data into memory allows
you to work with datasets that are too large to fit in memory all at once.
Detecting the artifacts in continuous data allows you to apply filters
(e.g. a band-pass filter to zoom in on the muscle artifacts on the temporal
channels) without having to worry about edge effects due to the filter
(i.e. filter ringing). Having the data in memory after segmenting/epoching is
however a very efficient way of browsing through the data which helps
in visualizing. So to conclude, there is not a single most optimal manner
to detect the artifacts: it just depends on the data properties and...
Can you write Python code for it?

import mne
from mne.datasets import sample
from mne.preprocessing import create_ecg_epochs, create_eog_epochs

# getting some data ready
data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_raw.fif'

raw = mne.io.read_raw_fif(raw_fname, preload=True)


I am working on the file "0.13/_downloads/plot_artifacts_detection.ipynb".
The first task is:
On MEG sensors we see narrow frequency peaks at 60, 120, 180, 240 Hz,
related to line noise.
But also some high amplitude signals between 25 and 32 Hz, hinting at other
biological artifacts such as ECG. These can be most easily detected in the
time domain using MNE helper functions
See tut_artifacts_filter.
ECG
finds ECG events, creates epochs, averages and plots
Can you write Python code for it?

average_ecg = create_ecg_epochs(raw).average()
print('We found %i ECG events' % average_ecg.nave)
average_ecg.plot_joint()


I am working on the file "0.13/_downloads/plot_artifacts_detection.ipynb".
The first task is:
we see high amplitude undulations in low frequencies, spanning across tens of
seconds
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
Accepting a shallower roll-off of the filter in the frequency domain makes
our time-domain response potentially much better. We end up with a
smoother slope through the transition region, but a much cleaner time
domain signal. Here again for the 1 sec filter:
Can you write Python code for it?

h = signal.firwin2(n, freq, gain, nyq=nyq)
plot_filter(h, 'Windowed 10-Hz transition (1.0 sec)', freq, gain)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
Filter it with a shallow cutoff, linear-phase FIR and compensate for
the delay:
Can you write Python code for it?

transition_band = 0.25 * f_p
f_s = f_p + transition_band
filter_dur = 6.6 / transition_band  # sec
n = int(sfreq * filter_dur)
freq = [0., f_p, f_s, sfreq / 2.]
gain = [1., 1., 0., 0.]
h = signal.firwin2(n, freq, gain, nyq=sfreq / 2.)
x_shallow = np.convolve(h, x)[len(h) // 2:]

plot_filter(h, 'MNE-Python 0.14 default', freq, gain)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
Finally, Let's also filter it with the
MNE-C default, which is a long-duration steep-slope FIR filter designed
using frequency-domain techniques:
Can you write Python code for it?

h = mne.filter.design_mne_c_filter(sfreq, l_freq=None, h_freq=f_p + 2.5)
x_mne_c = np.convolve(h, x)[len(h) // 2:]

transition_band = 5  # Hz (default in MNE-C)
f_s = f_p + transition_band
freq = [0., f_p, f_s, sfreq / 2.]
gain = [1., 1., 0., 0.]
plot_filter(h, 'MNE-C default', freq, gain)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
IIR filters
MNE-Python also offers IIR filtering functionality that is based on the
methods from :mod:scipy.signal. Specifically, we use the general-purpose
functions :func:scipy.signal.iirfilter and :func:scipy.signal.iirdesign,
which provide unified interfaces to IIR filter design.
Designing IIR filters
Let's continue with our design of a 40 Hz low-pass filter, and look at
some trade-offs of different IIR filters.
Often the default IIR filter is a Butterworth filter_, which is designed
to have a maximally flat pass-band. Let's look at a few orders of filter,
i.e., a few different number of coefficients used and therefore steepness
of the filter:
Can you write Python code for it?

sos = signal.iirfilter(2, f_p / nyq, btype='low', ftype='butter', output='sos')
plot_filter(sos, 'Butterworth order=2', freq, gain)

# Eventually this will just be from scipy signal.sosfiltfilt, but 0.18 is
# not widely adopted yet (as of June 2016), so we use our wrapper...
sosfiltfilt = mne.fixes.get_sosfiltfilt()
x_shallow = sosfiltfilt(sos, x)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
If we want a filter that is only 0.1 seconds long, we should probably use
something more like a 25 Hz transition band (0.2 sec = 5 cycles @ 25 Hz):
Can you write Python code for it?

trans_bandwidth = 25
f_s = f_p + trans_bandwidth
freq = [0, f_p, f_s, nyq]
h = signal.firwin2(n, freq, gain, nyq=nyq)
plot_filter(h, 'Windowed 50-Hz transition (0.2 sec)', freq, gain)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
In respose, Maess et al. 2016 [11]_ note that these simulations do not
address cases of pre-stimulus activity that is shared across conditions, as
applying baseline correction will effectively copy the topology outside the
baseline period. We can see this if we give our signal x with some
consistent pre-stimulus activity, which makes everything look bad.

Note

An important thing to keep in mind with these plots is that they
          are for a single simulated sensor. In multielectrode recordings
          the topology (i.e., spatial pattiern) of the pre-stimulus activity
          will leak into the post-stimulus period. This will likely create a
          spatially varying distortion of the time-domain signals, as the
          averaged pre-stimulus spatial pattern gets subtracted from the
          sensor time courses.

Putting some activity in the baseline period:
Can you write Python code for it?

n_pre = (t < 0).sum()
sig_pre = 1 - np.cos(2 * np.pi * np.arange(n_pre) / (0.5 * n_pre))
x[:n_pre] += sig_pre
baseline_plot(x)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
Let's make the stop-band tighter still with a longer filter (10 sec),
with a resulting larger x-axis:
Can you write Python code for it?

n = int(round(10. * sfreq)) + 1
t = np.arange(-n // 2, n // 2) / sfreq
h = np.sinc(2 * f_p * t) / (4 * np.pi)
plot_filter(h, 'Sinc (10.0 sec)', freq, gain)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
Now we have very sharp frequency suppression, but our filter rings for the
entire second. So this naïve method is probably not a good way to build
our low-pass filter.
Fortunately, there are multiple established methods to design FIR filters
based on desired response characteristics. These include:
1. The Remez_ algorithm (:func:`scipy.signal.remez`, `MATLAB firpm`_)
2. Windowed FIR design (:func:`scipy.signal.firwin2`, `MATLAB fir2`_)
3. Least squares designs (:func:`scipy.signal.firls`, `MATLAB firls`_)
4. Frequency-domain design (construct filter in Fourier
   domain and use an :func:`IFFT <scipy.fftpack.ifft>` to invert it)

Note

Remez and least squares designs have advantages when there are
          "do not care" regions in our frequency response. However, we want
          well controlled responses in all frequency regions.
          Frequency-domain construction is good when an arbitrary response
          is desired, but generally less clean (due to sampling issues) than
          a windowed approach for more straightfroward filter applications.
          Since our filters (low-pass, high-pass, band-pass, band-stop)
          are fairly simple and we require precisel control of all frequency
          regions, here we will use and explore primarily windowed FIR
          design.

If we relax our frequency-domain filter requirements a little bit, we can
use these functions to construct a lowpass filter that instead has a
transition band, or a region between the pass frequency $f_p$
and stop frequency $f_s$, e.g.:
Can you write Python code for it?

trans_bandwidth = 10  # 10 Hz transition band
f_s = f_p + trans_bandwidth  # = 50 Hz

freq = [0., f_p, f_s, nyq]
gain = [1., 1., 0., 0.]
ax = plt.subplots(1, figsize=half_height)[1]
plot_ideal(freq, gain, ax)
ax.set(title='%s Hz lowpass with a %s Hz transition' % (f_p, trans_bandwidth))
mne.viz.tight_layout()
plt.show()


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
Take for example an ideal low-pass filter, which would give a value of 1 in
the pass-band (up to frequency $f_p$) and a value of 0 in the stop-band
(down to frequency $f_s$) such that $f_p=f_s=40$ Hz here
(shown to a lower limit of -60 dB for simplicity):
Can you write Python code for it?

nyq = sfreq / 2.  # the Nyquist frequency is half our sample rate
freq = [0, f_p, f_p, nyq]
gain = [1, 1, 0, 0]

def box_off(ax):
    ax.grid(zorder=0)
    for key in ('top', 'right'):
        ax.spines[key].set_visible(False)

def plot_ideal(freq, gain, ax):
    freq = np.maximum(freq, xlim[0])
    xs, ys = list(), list()
    my_freq, my_gain = list(), list()
    for ii in range(len(freq)):
        xs.append(freq[ii])
        ys.append(ylim[0])
        if ii < len(freq) - 1 and gain[ii] != gain[ii + 1]:
            xs += [freq[ii], freq[ii + 1]]
            ys += [ylim[1]] * 2
            my_freq += np.linspace(freq[ii], freq[ii + 1], 20,
                                   endpoint=False).tolist()
            my_gain += np.linspace(gain[ii], gain[ii + 1], 20,
                                   endpoint=False).tolist()
        else:
            my_freq.append(freq[ii])
            my_gain.append(gain[ii])
    my_gain = 10 * np.log10(np.maximum(my_gain, 10 ** (ylim[0] / 10.)))
    ax.fill_between(xs, ylim[0], ys, color='r', alpha=0.1)
    ax.semilogx(my_freq, my_gain, 'r--', alpha=0.5, linewidth=4, zorder=3)
    xticks = [1, 2, 4, 10, 20, 40, 100, 200, 400]
    ax.set(xlim=xlim, ylim=ylim, xticks=xticks, xlabel='Frequency (Hz)',
           ylabel='Amplitude (dB)')
    ax.set(xticklabels=xticks)
    box_off(ax)

half_height = np.array(plt.rcParams['figure.figsize']) * [1, 0.5]
ax = plt.subplots(1, figsize=half_height)[1]
plot_ideal(freq, gain, ax)
ax.set(title='Ideal %s Hz lowpass' % f_p)
mne.viz.tight_layout()
plt.show()


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
The falloff of this filter is not very steep.

Warning

For brevity, we do not show the phase of these filters here.
             In the FIR case, we can design linear-phase filters, and
             compensate for the delay (making the filter acausal) if
             necessary. This cannot be done
             with IIR filters, as they have a non-linear phase.
             As the filter order increases, the
             phase distortion near and in the transition band worsens.
             However, if acausal (forward-backward) filtering can be used,
             e.g. with :func:`scipy.signal.filtfilt`, these phase issues
             can be mitigated.

Note

Here we have made use of second-order sections (SOS)
          by using :func:`scipy.signal.sosfilt` and, under the
          hood, :func:`scipy.signal.zpk2sos` when passing the
          ``output='sos'`` keyword argument to
          :func:`scipy.signal.iirfilter`. The filter definitions
          given in tut_filtering_basics_ use the polynomial
          numerator/denominator (sometimes called "tf") form ``(b, a)``,
          which are theoretically equivalent to the SOS form used here.
          In practice, however, the SOS form can give much better results
          due to issues with numerical precision (see
          :func:`scipy.signal.sosfilt` for an example), so SOS should be
          used when possible to do IIR filtering.

Let's increase the order, and note that now we have better attenuation,
with a longer impulse response:
Can you write Python code for it?

sos = signal.iirfilter(8, f_p / nyq, btype='low', ftype='butter', output='sos')
plot_filter(sos, 'Butterworth order=8', freq, gain)
x_steep = sosfiltfilt(sos, x)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
There are other types of IIR filters that we can use. For a complete list,
check out the documentation for :func:scipy.signal.iirdesign. Let's
try a Chebychev (type I) filter, which trades off ripple in the pass-band
to get better attenuation in the stop-band:
Can you write Python code for it?

sos = signal.iirfilter(8, f_p / nyq, btype='low', ftype='cheby1', output='sos',
                       rp=1)  # dB of acceptable pass-band ripple
plot_filter(sos, 'Chebychev-1 order=8, ripple=1 dB', freq, gain)


I am working on the file "0.13/_downloads/plot_background_filtering.ipynb".
The first task is:
This filter hypothetically achieves zero ripple in the frequency domain,
perfect attenuation, and perfect steepness. However, due to the discontunity
in the frequency response, the filter would require infinite ringing in the
time domain (i.e., infinite order) to be realized. Another way to think of
this is that a rectangular window in frequency is actually sinc_ function
in time, which requires an infinite number of samples, and thus infinite
time, to represent. So although this filter has ideal frequency suppression,
it has poor time-domain characteristics.
Let's try to naïvely make a brick-wall filter of length 0.1 sec, and look
at the filter itself in the time domain and the frequency domain:
Can you write Python code for it?

n = int(round(0.1 * sfreq)) + 1
t = np.arange(-n // 2, n // 2) / sfreq  # center our sinc
h = np.sinc(2 * f_p * t) / (4 * np.pi)

def plot_filter(h, title, freq, gain, show=True):
    if h.ndim == 2:  # second-order sections
        sos = h
        n = mne.filter.estimate_ringing_samples(sos)
        h = np.zeros(n)
        h[0] = 1
        h = signal.sosfilt(sos, h)
        H = np.ones(512, np.complex128)
        for section in sos:
            f, this_H = signal.freqz(section[:3], section[3:])
            H *= this_H
    else:
        f, H = signal.freqz(h)
    fig, axs = plt.subplots(2)
    t = np.arange(len(h)) / sfreq
    axs[0].plot(t, h, color=blue)
    axs[0].set(xlim=t[[0, -1]], xlabel='Time (sec)',
               ylabel='Amplitude h(n)', title=title)
    box_off(axs[0])
    f *= sfreq / (2 * np.pi)
    axs[1].semilogx(f, 10 * np.log10((H * H.conj()).real), color=blue,
                    linewidth=2, zorder=4)
    plot_ideal(freq, gain, axs[1])
    mne.viz.tight_layout()
    if show:
        plt.show()

plot_filter(h, 'Sinc (0.1 sec)', freq, gain)


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Brainstorm auditory tutorial dataset
Here we compute the evoked from raw for the auditory Brainstorm
tutorial dataset. For comparison, see [1]_ and:
http://neuroimage.usc.edu/brainstorm/Tutorials/Auditory

Experiment:
    - One subject, 2 acquisition runs 6 minutes each.
    - Each run contains 200 regular beeps and 40 easy deviant beeps.
    - Random ISI: between 0.7s and 1.7s seconds, uniformly distributed.
    - Button pressed when detecting a deviant with the right index finger.
The specifications of this dataset were discussed initially on the
FieldTrip bug tracker <http://bugzilla.fcdonders.nl/show_bug.cgi?id=2300>_.
References
.. [1] Tadel F, Baillet S, Mosher JC, Pantazis D, Leahy RM.
       Brainstorm: A User-Friendly Application for MEG/EEG Analysis.
       Computational Intelligence and Neuroscience, vol. 2011, Article ID
       879716, 13 pages, 2011. doi:10.1155/2011/879716
Can you write Python code for it?

# Authors: Mainak Jas <mainak.jas@telecom-paristech.fr>
#          Eric Larson <larson.eric.d@gmail.com>
#          Jaakko Leppakangas <jaeilepp@student.jyu.fi>
#
# License: BSD (3-clause)

import os.path as op
import pandas as pd
import numpy as np

import mne
from mne import combine_evoked
from mne.minimum_norm import apply_inverse
from mne.datasets.brainstorm import bst_auditory
from mne.io import read_raw_ctf
from mne.filter import notch_filter, low_pass_filter

print(__doc__)


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Epoching and averaging.
First some parameters are defined and events extracted from the stimulus
channel (UPPT001). The rejection thresholds are defined as peak-to-peak
values and are in T / m for gradiometers, T for magnetometers and
V for EOG and EEG channels.
Can you write Python code for it?

tmin, tmax = -0.1, 0.5
event_id = dict(standard=1, deviant=2)
reject = dict(mag=4e-12, eog=250e-6)
# find events
events = mne.find_events(raw, stim_channel='UPPT001')


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Here we plot the ERF of standard and deviant conditions. In both conditions
we can see the P50 and N100 responses. The mismatch negativity is visible
only in the deviant condition around 100-200 ms. P200 is also visible around
170 ms in both conditions but much stronger in the standard condition. P300
is visible in deviant condition only (decision making in preparation of the
button press). You can view the topographies from a certain time span by
painting an area with clicking and holding the left mouse button.
Can you write Python code for it?

evoked_std.plot(window_title='Standard', gfp=True)
evoked_dev.plot(window_title='Deviant', gfp=True)


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Show activations as topography figures.
Can you write Python code for it?

times = np.arange(0.05, 0.301, 0.025)
evoked_std.plot_topomap(times=times, title='Standard')
evoked_dev.plot_topomap(times=times, title='Deviant')


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Source estimation.
We compute the noise covariance matrix from the empty room measurement
and use it for the other runs.
Can you write Python code for it?

reject = dict(mag=4e-12)
cov = mne.compute_raw_covariance(raw_erm, reject=reject)
cov.plot(raw_erm.info)
del raw_erm


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
The epochs (trials) are created for MEG channels. First we find the picks
for MEG and EOG channels. Then the epochs are constructed using these picks.
The epochs overlapping with annotated bad segments are also rejected by
default. To turn off rejection by bad segments (as was done earlier with
saccades) you can use keyword reject_by_annotation=False.
Can you write Python code for it?

picks = mne.pick_types(raw.info, meg=True, eeg=False, stim=False, eog=True,
                       exclude='bads')

epochs = mne.Epochs(raw, events, event_id, tmin, tmax, picks=picks,
                    baseline=(None, 0), reject=reject, preload=False,
                    proj=True)


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
The sources are computed using dSPM method and plotted on an inflated brain
surface. For interactive controls over the image, use keyword
time_viewer=True.
Standard condition.
Can you write Python code for it?

stc_standard = mne.minimum_norm.apply_inverse(evoked_std, inv, lambda2, 'dSPM')
brain = stc_standard.plot(subjects_dir=subjects_dir, subject=subject,
                          surface='inflated', time_viewer=False, hemi='lh',
                          initial_time=0.1, time_unit='s')
del stc_standard, brain


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
To reduce memory consumption and running time, some of the steps are
precomputed. To run everything from scratch change this to False. With
use_precomputed = False running time of this script can be several
minutes even on a fast computer.
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
To save time and memory, the forward solution is read from a file. Set
use_precomputed=False in the beginning of this script to build the
forward solution from scratch. The head surfaces for constructing a BEM
solution are read from a file. Since the data only contains MEG channels, we
only need the inner skull surface for making the forward solution. For more
information: CHDBBCEJ, :func:mne.setup_source_space,
create_bem_model, :func:mne.bem.make_watershed_bem.
Can you write Python code for it?

if use_precomputed:
    fwd_fname = op.join(data_path, 'MEG', 'bst_auditory',
                        'bst_auditory-meg-oct-6-fwd.fif')
    fwd = mne.read_forward_solution(fwd_fname)
else:
    src = mne.setup_source_space(subject, spacing='ico4',
                                 subjects_dir=subjects_dir, overwrite=True)
    model = mne.make_bem_model(subject=subject, ico=4, conductivity=[0.3],
                               subjects_dir=subjects_dir)
    bem = mne.make_bem_solution(model)
    fwd = mne.make_forward_solution(evoked_std.info, trans=trans, src=src,
                                    bem=bem)

inv = mne.minimum_norm.make_inverse_operator(evoked_std.info, fwd, cov)
snr = 3.0
lambda2 = 1.0 / snr ** 2
del fwd


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Typical preprocessing step is the removal of power line artifact (50 Hz or
60 Hz). Here we notch filter the data at 60, 120 and 180 to remove the
original 60 Hz artifact and the harmonics. Normally this would be done to
raw data (with :func:mne.io.Raw.filter), but to reduce memory consumption
of this tutorial, we do it at evoked stage.
Can you write Python code for it?

if use_precomputed:
    sfreq = evoked_std.info['sfreq']
    nchan = evoked_std.info['nchan']
    notches = [60, 120, 180]
    for ch_idx in range(nchan):
        evoked_std.data[ch_idx] = notch_filter(evoked_std.data[ch_idx], sfreq,
                                               notches, verbose='ERROR')
        evoked_dev.data[ch_idx] = notch_filter(evoked_dev.data[ch_idx], sfreq,
                                               notches, verbose='ERROR')
        evoked_std.data[ch_idx] = low_pass_filter(evoked_std.data[ch_idx],
                                                  sfreq, 100, verbose='ERROR')
        evoked_dev.data[ch_idx] = low_pass_filter(evoked_dev.data[ch_idx],
                                                  sfreq, 100, verbose='ERROR')


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Typical preprocessing step is the removal of power line artifact (50 Hz or
60 Hz). Here we notch filter the data at 60, 120 and 180 to remove the
original 60 Hz artifact and the harmonics. The power spectra are plotted
before and after the filtering to show the effect. The drop after 600 Hz
appears because the data was filtered during the acquisition. In memory
saving mode we do the filtering at evoked stage, which is not something you
usually would do.
Can you write Python code for it?

if not use_precomputed:
    meg_picks = mne.pick_types(raw.info, meg=True, eeg=False)
    raw.plot_psd(picks=meg_picks)
    notches = np.arange(60, 181, 60)
    raw.notch_filter(notches)
    raw.plot_psd(picks=meg_picks)


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
Visually inspect the effects of projections. Click on 'proj' button at the
bottom right corner to toggle the projectors on/off. EOG events can be
plotted by adding the event list as a keyword argument. As the bad segments
and saccades were added as annotations to the raw data, they are plotted as
well.
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
We also lowpass filter the data at 100 Hz to remove the hf components.
Can you write Python code for it?

if not use_precomputed:
    raw.filter(None, 100., h_trans_bandwidth=0.5, filter_length='10s',
               phase='zero-double')


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
We can see the MMN effect more clearly by looking at the difference between
the two conditions. P50 and N100 are no longer visible, but MMN/P200 and
P300 are emphasised.
Can you write Python code for it?

evoked_difference = combine_evoked([evoked_dev, -evoked_std], weights='equal')
evoked_difference.plot(window_title='Difference', gfp=True)


I am working on the file "0.13/_downloads/plot_brainstorm_auditory.ipynb".
The first task is:
We only use first 40 good epochs from each run. Since we first drop the bad
epochs, the indices of the epochs are no longer same as in the original
epochs collection. Investigation of the event timings reveals that first
epoch from the second run corresponds to index 182.
Can you write Python code for it?

epochs.drop_bad()
epochs_standard = mne.concatenate_epochs([epochs['standard'][range(40)],
                                          epochs['standard'][182:222]])
epochs_standard.load_data()  # Resampling to save memory.
epochs_standard.resample(600, npad='auto')
epochs_deviant = epochs['deviant'].load_data()
epochs_deviant.resample(600, npad='auto')
del epochs, picks


I am working on the file "0.13/_downloads/plot_brainstorm_phantom_ctf.ipynb".
The first task is:
Brainstorm CTF phantom tutorial dataset
Here we compute the evoked from raw for the Brainstorm CTF phantom
tutorial dataset. For comparison, see [1]_ and:
http://neuroimage.usc.edu/brainstorm/Tutorials/PhantomCtf

References
.. [1] Tadel F, Baillet S, Mosher JC, Pantazis D, Leahy RM.
       Brainstorm: A User-Friendly Application for MEG/EEG Analysis.
       Computational Intelligence and Neuroscience, vol. 2011, Article ID
       879716, 13 pages, 2011. doi:10.1155/2011/879716
Can you write Python code for it?

# Authors: Eric Larson <larson.eric.d@gmail.com>
#
# License: BSD (3-clause)

import os.path as op
import numpy as np
import matplotlib.pyplot as plt

import mne
from mne import fit_dipole
from mne.datasets.brainstorm import bst_phantom_ctf
from mne.io import read_raw_ctf

print(__doc__)


I am working on the file "0.13/_downloads/plot_brainstorm_phantom_ctf.ipynb".
The first task is:
But here we can get slightly better noise suppression, lower localization
bias, and a better dipole goodness of fit with spatio-temporal (tSSS)
Maxwell filtering:
Can you write Python code for it?

raw.apply_gradient_compensation(0)  # must un-do software compensation first
mf_kwargs = dict(origin=(0., 0., 0.), st_duration=10.)
raw = mne.preprocessing.maxwell_filter(raw, **mf_kwargs)
raw.plot()


I am working on the file "0.13/_downloads/plot_brainstorm_phantom_ctf.ipynb".
The first task is:
Let's create some events using this signal by thresholding the sinusoid.
Can you write Python code for it?

events = np.where(np.diff(sinusoid > 0.5) > 0)[1] + raw.first_samp
events = np.vstack((events, np.zeros_like(events), np.ones_like(events))).T


I am working on the file "0.13/_downloads/plot_brainstorm_phantom_ctf.ipynb".
The first task is:
The CTF software compensation works reasonably well:
Can you write Python code for it?


I am working on the file "0.13/_downloads/plot_brainstorm_phantom_ctf.ipynb".
The first task is:
The data were collected with a CTF system at 2400 Hz.
Can you write Python code for it?

data_path = bst_phantom_ctf.data_path()

# Switch to these to use the higher-SNR data:
# raw_path = op.join(data_path, 'phantom_200uA_20150709_01.ds')
# dip_freq = 7.
raw_path = op.join(data_path, 'phantom_20uA_20150603_03.ds')
dip_freq = 23.
erm_path = op.join(data_path, 'emptyroom_20150709_01.ds')
raw = read_raw_ctf(raw_path, preload=True)