Repository: mne-tools/mne-python

Issue:
BUG: Intro example uses too much memory
Running:
```
$ mprof run plot_introduction.py
$ mprof plot
```
I get:

![Screenshot from 2019-05-15 14-44-16](https://user-images.githubusercontent.com/2365790/57800689-00f18d80-7720-11e9-8aa7-006621cdb9b8.png)

Examples that use over ~1.5 GB (and 1.0 GB is a safer goal) are prone to die on CircleCI, e.g.:

https://circleci.com/gh/mne-tools/mne-python/13124

@drammock can you find a way to cut down the memory usage? Often cropping to a shorter duration (e.g., 60 sec) is enough. Or using `preload=False` where possible. Also a new note of this loose requirement in the revamped dev docs might help.


Discussion:
I'll take a look.  I'm already using `del` and avoiding `preload` where I can, but I can probably find ways to deflate it further.
For example adding `stop=30` to the `ica.fit` line gets rid of the first big bump:

![Screenshot from 2019-05-15 15-15-58](https://user-images.githubusercontent.com/2365790/57802606-5cbe1580-7724-11e9-87ea-557f2e3347a5.png)

The others probably come from loading the raw data, then applying ICA to it. Both can be alleviated by doing `raw.crop(0, 60)` with a suitable message at the beginning that it's done just to save memory. Doing this on line 71 I get the following memory plot (no idea if the plots are actually still reasonable):

![Screenshot from 2019-05-15 15-19-18](https://user-images.githubusercontent.com/2365790/57802821-d2c27c80-7724-11e9-8c42-b5c4993e335d.png)

don't use maybe sample_audvis_raw.fif but the filtered version so you can decimate? you should also use decim in ICA rather than crop.
> you should also use decim in ICA rather than crop.

I'd rather make this a note in the example as an option for users, rather than actually doing it. Adding `raw.crop(0, 60)` makes the entire example runnable from a memory perspective. Without it (even with `decim` or `stop` in `ica.fit`), we will need a number of additional (possibly more complicated) patches elsewhere to avoid excessive memory consumption.
try with crop(0, 180) as it should still provide OK results/images. I am
still a bit puzzled why this happens
given that so many examples should do such things.

>

> but the filtered version so you can decimate

`sample_audvis_filt-0-40_raw.fif` is actually both filtered *and* decimated already. It does give good memory performance, just swapping in that filename gives:

![Screenshot from 2019-05-15 15-36-24](https://user-images.githubusercontent.com/2365790/57803983-a1977b80-7727-11e9-912b-c7f67b324e0a.png)

> try with crop(0, 180)

Using the standard raw file and doing `raw.crop(0, 180)` gives:

![Screenshot from 2019-05-15 15-39-42](https://user-images.githubusercontent.com/2365790/57804023-baa02c80-7727-11e9-89c1-de151a258a16.png)

Probably not good enough, and if we're going to crop at all we might as well get better memory with `.crop(0, 60)`.

> still a bit puzzled why this happens
> given that so many examples should do such things.

We almost always crop, do `ica.fit(..., stop=...)`, or use the filtered+decimated file.


it's hard to evaluate as reducing memory is not the end goal. We need
to make sure
that the analysis results still look correct. I cannot judge this with
just these plots.

I'm rebuilding the docs locally after swapping in the `filt-0-40` file. If the plots still look good I'll open a PR. I don't like the `crop` approach because it makes some of the plots look crappy.
> it's hard to evaluate as reducing memory is not the end goal. We need to make sure that the analysis results still look correct. I cannot judge this with just these plots.

Agreed, these plots do not provide a sufficient condition, just a necessary one. So we should only consider `.crop(0, 60)` (probably also `.crop(0, 120)`) or using the 0-40 file.

Resolving patch:
diff --git a/tutorials/intro/plot_introduction.py b/tutorials/intro/plot_introduction.py
--- a/tutorials/intro/plot_introduction.py
+++ b/tutorials/intro/plot_introduction.py
@@ -40,19 +40,24 @@
 # automatically download the dataset if it isn't found in one of the expected
 # locations, then return the directory path to the dataset (see the
 # documentation of :func:`~mne.datasets.sample.data_path` for a list of places
-# it checks before downloading).
+# it checks before downloading). Note also that for this tutorial to run
+# smoothly on our servers, we're using a filtered and downsampled version of
+# the data (:file:`sample_audvis_filt-0-40_raw.fif`), but an unfiltered version
+# (:file:`sample_audvis_raw.fif`) is also included in the sample dataset and
+# could be substituted here when running the tutorial locally.
 
 sample_data_folder = mne.datasets.sample.data_path()
 sample_data_raw_file = os.path.join(sample_data_folder, 'MEG', 'sample',
-                                    'sample_audvis_raw.fif')
+                                    'sample_audvis_filt-0-40_raw.fif')
 raw = mne.io.read_raw_fif(sample_data_raw_file)
 
 ###############################################################################
 # By default, :func:`~mne.io.read_raw_fif` displays some information about the
-# file it's loading; for example, here it tells us that there are three
+# file it's loading; for example, here it tells us that there are four
 # "projection items" in the file along with the recorded data; those are
 # :term:`SSP projectors <projector>` calculated to remove environmental noise
-# from the MEG signals, and are discussed
+# from the MEG signals, plus a projector to mean-reference the EEG channels;
+# these are discussed
 # in a later tutorial. In addition to the information displayed during loading,
 # you can get a glimpse of the basic details of a :class:`~mne.io.Raw` object
 # by printing it; even more is available by printing its ``info`` attribute
@@ -71,13 +76,14 @@
 
 ###############################################################################
 # :class:`~mne.io.Raw` objects also have several built-in plotting methods;
-# here we show the power spectral density for each sensor type with
+# here we show the power spectral density (PSD) for each sensor type with
 # :meth:`~mne.io.Raw.plot_psd`, as well as a plot of the raw sensor traces with
-# :meth:`~mne.io.Raw.plot`. In interactive Python sessions,
-# :meth:`~mne.io.Raw.plot` is interactive and allows scrolling, scaling, bad
-# channel marking, annotation, projector toggling, etc.
+# :meth:`~mne.io.Raw.plot`. In the PSD plot, we'll only plot frequencies below
+# 50 Hz (since our data are low-pass filtered at 40 Hz). In interactive Python
+# sessions, :meth:`~mne.io.Raw.plot` is interactive and allows scrolling,
+# scaling, bad channel marking, annotation, projector toggling, etc.
 
-raw.plot_psd()
+raw.plot_psd(fmax=50)
 raw.plot(duration=5, n_channels=30)
 
 ###############################################################################


Test patch: