Repository: sphinx-gallery/sphinx-gallery

Issue:
JupyterLab support on binder
I just heard about urlpath=lab. Maybe it would be nice to support at one point. I could not find a way to point to a given file with JupyterLab, i.e. using both urlpath=lab&filepath=myipnb. Is this correct @choldgraf?

Thinking about it out loud, in a multi-notebook context (at the moment we are just opening a single notebook in binder but you could imagine pointing to a folder e.g. in the main gallery or it seems like urlpath=lab works like this by default) it feels like the notebooks should be in auto_examples as well as in _downloads. This way the notebook folder structure mirrors the example folder (maybe that does get rid of the duplicated filename problem).

I am not sure why .ipynb can not be seen in https://github.com/sphinx-gallery/sphinx-gallery.github.io/tree/master/auto_examples but this can probably be fixed (at the cost of duplicating .ipynb in the repo, once in _downloads, once in auto_examples which is probably fine ...).

Discussion:
lesteve: Thanks a lot for the PR! I have not looked at the details yet, but the main question I have is: is there a good reason that we *have* to use `_static`? I would have thought that the variable should be called `notebooks_output_folder` or something similar and that and that the user could chose any folder he wants but I may be missing a sphinx quirk.
lesteve: Also some CI failures seem genuine, but I don't think that is the main problem right now.

choldgraf: @lesteve as far as I could figure out, sphinx won't copy over anything that isn't RST or markdown (e.g., it doesn't copy over anything other than this in the "auto_examples" folder, but correct me if I'm wrong) so the only option is to put it in `_static/myfolder`

If somebody knows a hack around this, lemme know!
lesteve: Can we not copy the notebooks ourselves? This was my naive strategy originally:
* we may be able to know the html output folder (`_build/html` in the sphinx-gallery case): if `app` is a `Sphinx` object, it looks like `app.outdir` is what we want.
* once we know the html output folder we add `notebooks_output_folder` to it so that the notebook are inside the generated html. That means that the notebooks are part of the repo and we can use binder with the appropriate URLs.
lesteve: Something like this, copying the notebooks rather than moving seems safer. Just to be clear, in my mind `notebook_output_dir` is a subfolder or `app.outdir`. Just to take an example in the sphix-gallery case, if I have `notebook_output_dir='my/notebooks'` I should have the notebooks in `_build/html/my/notebooks`.

I agree that this is a bit working around sphinx but I feel this is a bit what sphinx-gallery is already doing to some respect (e.g. the backreferences aka mini-galleries in the reference API doc comes to mind). Of course, if there are ways to do it the sphinx way for this PR or others, this would be more than welcome!
choldgraf: lemme give this a shot and we'll see how it looks :-)
codecov-io: # [Codecov](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371?src=pr&el=h1) Report
> Merging [#371](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371?src=pr&el=desc) into [master](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/commit/c9817bf14c59f211574e1b3d481ee9353ead3674?src=pr&el=desc) will **decrease** coverage by `3.28%`.
> The diff coverage is `84.44%`.

[![Impacted file tree graph](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/graphs/tree.svg?token=K4eytvhwU5&src=pr&height=150&width=650)](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #371      +/-   ##
==========================================
- Coverage   92.07%   88.78%   -3.29%     
==========================================
  Files          27       27              
  Lines        1842     1873      +31     
==========================================
- Hits         1696     1663      -33     
- Misses        146      210      +64
```


| [Impacted Files](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sphinx\_gallery/tests/test\_binder.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvdGVzdHMvdGVzdF9iaW5kZXIucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [sphinx\_gallery/utils.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvdXRpbHMucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [sphinx\_gallery/gen\_rst.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvZ2VuX3JzdC5weQ==) | `87.57% <40%> (-8.11%)` | :arrow_down: |
| [sphinx\_gallery/binder.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvYmluZGVyLnB5) | `82.08% <93.33%> (+2.08%)` | :arrow_up: |
| [sphinx\_gallery/tests/test\_gen\_rst.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvdGVzdHMvdGVzdF9nZW5fcnN0LnB5) | `90.78% <0%> (-7.84%)` | :arrow_down: |
| [sphinx\_gallery/tests/test\_docs\_resolv.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvdGVzdHMvdGVzdF9kb2NzX3Jlc29sdi5weQ==) | `93.93% <0%> (-6.07%)` | :arrow_down: |
| [sphinx\_gallery/py\_source\_parser.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvcHlfc291cmNlX3BhcnNlci5weQ==) | `76.54% <0%> (-4.94%)` | :arrow_down: |
| [sphinx\_gallery/tests/test\_notebook.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvdGVzdHMvdGVzdF9ub3RlYm9vay5weQ==) | `95.34% <0%> (-4.66%)` | :arrow_down: |
| [sphinx\_gallery/backreferences.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvYmFja3JlZmVyZW5jZXMucHk=) | `84.82% <0%> (-4.47%)` | :arrow_down: |
| ... and [2 more](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371?src=pr&el=footer). Last update [c9817bf...69c8639](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/371?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

choldgraf: @lesteve what do you think about the latest push? You can now specify an arbitrary folder, and the notebooks will be copied to that folder within `_build`.

e.g.:

http://predictablynoisy.com/sphinx-gallery/auto_examples/plot_seaborn.html#sphx-glr-auto-examples-plot-seaborn-py

I think part of the errors are because I'm using a python 3 function somewhere, I'll try and run that down, but in the meantime WDYT about the current pattern?

Also - do you think that we should actually do this automatically, rather than have people provide a config to enable it? E.g., we could basically make `notebooks_dir` to default to `notebooks`.

choldgraf: ok I *think* the latest push should address much of @lesteve 's concerns. It factors out the notebook copying step to the very end as you suggested. Also uses `os.walk` now, so perhaps that means the python 2 builds will be happy again
choldgraf: oops, that was @Titan-C 's concerns not @lesteve 's concerns :-)

Latest version sets `notebooks_folder` to default to `notebooks/`, and lets people change this if they wish.

LMK if there's anything else I should tackle other than making tests pass.
choldgraf: OK I think we're looking good w/ tests and coverage. This is ready for another review, or ready to 🚢 
choldgraf: huh - yes I see the same thing too! I didn't check to see whether *extra* files and folders were created haha...lemme check on this. And I agree with `_dir` being consistent with the broader project
choldgraf: OK I think I fixed it up. Here's the my build tree now:

```
doc/_build/html/notebooks/
├── auto_examples
│   ├── no_output
│   │   ├── just_code.ipynb
│   │   ├── plot_raise.ipynb
│   │   ├── plot_strings.ipynb
│   │   └── plot_syntaxerror.ipynb
│   ├── plot_choose_thumbnail.ipynb
│   ├── plot_colors.ipynb
│   ├── plot_exp.ipynb
│   ├── plot_function_identifier.ipynb
│   ├── plot_gallery_version.ipynb
│   ├── plot_quantum.ipynb
│   ├── plot_seaborn.ipynb
│   ├── plot_sys_argv.ipynb
│   ├── plot_unicode_everywhere.ipynb
│   └── sin_func
│       ├── plot_sin_black_background.ipynb
│       └── plot_sin.ipynb
└── tutorials
    ├── plot_notebook.ipynb
    ├── plot_parse.ipynb
    └── seaborn
        └── plot_seaborn_notebook.ipynb
```

also replaced `_folder` with `_dir`
choldgraf: friendly ping :-)
Titan-C: This looks good to me. Thank you for the quick refactoring. Happy to see you use `relpath` now and not your own function. It makes the code more readable now. I took the liberty to update the function that copies the jupyter notebooks. Please review it.
This is ready for a merge if everything looks good on @choldgraf live website, then we can restore the old config.
choldgraf: Thanks for the patch! I cleaned up the filtering function a little bit since the multi-part boolean logic was confusing to me. Other than that, I think this is ready to go from my end. Here's a link to the live build:

https://predictablynoisy.com/sphinx-gallery/auto_examples/sin_func/plot_sin.html#sphx-glr-auto-examples-sin-func-plot-sin-py
choldgraf: Sounds good - I am all for beautifully complex list comprehensions, but I must admit it took me like 10 minutes to figure out what that one was doing :-) 

I think the main thing that confused me was these lines:

`entry.endswith('.ipynb') != os.path.isdir(os.path.join(path, entry))`

anyway feel free to suggest alternate namings etc...
Titan-C: > `entry.endswith('.ipynb') != os.path.isdir(os.path.join(path, entry))`

You have to see != as an exclusive OR. Then it works fine and you can
read: either a Jupyter notebook or a directory. Then the NOT in front
transforms it to neither ... nor.

choldgraf: I tried updating the functions name/description/structure to make it a bit more clear...LMK what you think (or feel free to make edits)
lesteve: Sorry I haven't had time to look at this which I think is really important. I'll try to find some time beginning of next week.

Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 6630556..3b90073 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -7,9 +7,10 @@ git master
 New features
 ''''''''''''
 
-* Added experimental support to auto-generate Binder links for examples via ``binder``
-  config. Note that this API may change in the future. `#244
-  <https://github.com/sphinx-gallery/sphinx-gallery/pull/244>`_
+* Added experimental support to auto-generate Binder links for examples via
+  ``binder`` config. Note that this API may change in the future. `#244
+  <https://github.com/sphinx-gallery/sphinx-gallery/pull/244>`_ and `#371
+  <https://github.com/sphinx-gallery/sphinx-gallery/pull/371>`_.
 * **CSS** Download and binder buttons are now on the right side for large
   screens. This might change downstream CSS layouts.
   "div.sphx-glr-footer-example" can be used to capture these specific
diff --git a/doc/advanced_configuration.rst b/doc/advanced_configuration.rst
index 7fbe48b..ae46915 100644
--- a/doc/advanced_configuration.rst
+++ b/doc/advanced_configuration.rst
@@ -24,7 +24,7 @@ file:
 - ``default_thumb_file`` (:ref:`custom_default_thumb`)
 - ``thumbnail_size`` (:ref:`setting_thumbnail_size`)
 - ``line_numbers`` (:ref:`adding_line_numbers`)
-- ``download_section_examples`` (:ref:`disable_joint_download`)
+- ``download_all_examples`` (:ref:`disable_all_scripts_download`)
 - ``plot_gallery`` (:ref:`without_execution`)
 - ``find_mayavi_figures`` (:ref:`find_mayavi`)
 - ``abort_on_example_error`` (:ref:`abort_on_first`)
@@ -381,19 +381,19 @@ Note that for Sphinx < 1.3, the line numbers will not be consistent with the
 original file.
 
 
-.. _disable_joint_download:
+.. _disable_all_scripts_download:
 
-Disabling joint download of scripts
-===================================
+Disabling download button of all scripts
+========================================
 
-By default Sphinx-Gallery prepares zip files of all python scripts and
-all Jupyter notebooks for each gallery section and makes them
-available for download at the end of each section. To disable this
-behavior add to the configuration dictionary in your ``conf.py`` file::
+By default Sphinx-Gallery collects all python scripts and all Jupyter
+notebooks from each gallery into zip files which are made available for
+download at the bottom of each gallery. To disable this behavior add to the
+configuration dictionary in your ``conf.py`` file::
 
     sphinx_gallery_conf = {
         ...
-        'download_section_examples': False,
+        'download_all_examples': False,
     }
 
 
@@ -442,19 +442,27 @@ dictionary following the pattern below::
     sphinx_gallery_conf = {
       ...
       'binder': {
+         # Required keys
          'org': '<github_org>',
          'repo': '<github_repo>',
          'url': '<binder_url>',  # Any URL of a binder server. Must be full URL (e.g. https://mybinder.org).
          'branch': '<branch-for-documentation>',  # Can be any branch, tag, or commit hash. Use a branch that hosts your docs.
          'dependencies': '<list_of_paths_to_dependency_files>',
-         'filepath_prefix': '<prefix>' # Optional, a prefix to append to any filepaths in Binder links.
-                            use this if you move your built documentation to a sub-folder of your repository (e.g., "v2.1")
+         # Optional keys
+         'filepath_prefix': '<prefix>' # A prefix to append to any filepaths in Binder links.
+         'notebooks_dir': '<notebooks-directory-name>' # Jupyter notebooks for Binder will be copied to this directory (relative to site root).
+         'use_jupyter_lab': <bool> # Whether Binder links should start Jupyter Lab instead of the Jupyter Notebook interface.
          }
     }
 
 Note that ``branch:`` should be the branch on which your documentation is hosted.
 If you host your documentation on GitHub, this is usually ``gh-pages`` or ``master``.
 
+Each generated Jupyter Notebook will be copied to the folder
+specified in ``notebooks_dir``. This will be a subfolder of the sphinx output
+directory and included with your site build.
+Binder links will point to these notebooks.
+
 .. important::
 
    ``dependencies`` should be a list of paths to Binder configuration files that
diff --git a/doc/conf.py b/doc/conf.py
index ac53cf6..773afba 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -345,6 +345,8 @@ sphinx_gallery_conf = {
                'repo': 'sphinx-gallery.github.io',
                'url': 'https://mybinder.org',
                'branch': 'master',
-               'dependencies': './binder/requirements.txt'
+               'dependencies': './binder/requirements.txt',
+               'notebooks_dir': 'notebooks',
+               'use_jupyter_lab': True,
                }
 }
diff --git a/sphinx_gallery/binder.py b/sphinx_gallery/binder.py
index 961a2be..7dced88 100644
--- a/sphinx_gallery/binder.py
+++ b/sphinx_gallery/binder.py
@@ -15,7 +15,7 @@ change in the future.
 
 """
 
-import shutil as sh
+import shutil
 import os
 
 try:
@@ -25,9 +25,13 @@ except NameError:
     unicode = str
 
 from .utils import replace_py_ipynb
+from . import sphinx_compatibility
 
 
-def gen_binder_url(fname, binder_conf):
+logger = sphinx_compatibility.getLogger('sphinx-gallery')
+
+
+def gen_binder_url(fname, binder_conf, gallery_conf):
     """Generate a Binder URL according to the configuration in conf.py.
 
     Parameters
@@ -45,20 +49,36 @@ def gen_binder_url(fname, binder_conf):
     """
     # Build the URL
     fpath_prefix = binder_conf.get('filepath_prefix')
-    binder_fpath = '_downloads/{}'.format(replace_py_ipynb(fname))
+    link_base = binder_conf.get('notebooks_dir')
+
+    # We want to keep the relative path to sub-folders
+    relative_link = os.path.relpath(fname, gallery_conf['src_dir'])
+    path_link = os.path.join(
+        link_base, replace_py_ipynb(relative_link))
+
+    # In case our website is hosted in a sub-folder
     if fpath_prefix is not None:
-        binder_fpath = '/'.join([fpath_prefix.strip('/'), binder_fpath])
+        path_link = '/'.join([fpath_prefix.strip('/'), path_link])
+
+    # Make sure we have the right slashes (in case we're on Windows)
+    path_link = path_link.replace(os.path.sep, '/')
+
+    # Create the URL
     binder_url = binder_conf['url']
     binder_url = '/'.join([binder_conf['url'],
                            'v2', 'gh',
                            binder_conf['org'],
                            binder_conf['repo'],
                            binder_conf['branch']])
-    binder_url += '?filepath={}'.format(binder_fpath)
+
+    if binder_conf.get('use_jupyter_lab', False) is True:
+        binder_url += '?urlpath=lab/tree/{}'.format(path_link)
+    else:
+        binder_url += '?filepath={}'.format(path_link)
     return binder_url
 
 
-def gen_binder_rst(fname, binder_conf):
+def gen_binder_rst(fname, binder_conf, gallery_conf):
     """Generate the RST + link for the Binder badge.
 
     Parameters
@@ -84,7 +104,8 @@ def gen_binder_rst(fname, binder_conf):
     rst : str
         The reStructuredText for the Binder badge that links to this file.
     """
-    binder_url = gen_binder_url(fname, binder_conf)
+    binder_conf = check_binder_conf(binder_conf)
+    binder_url = gen_binder_url(fname, binder_conf, gallery_conf)
 
     rst = (
         "\n"
@@ -95,17 +116,75 @@ def gen_binder_rst(fname, binder_conf):
     return rst
 
 
-def copy_binder_reqs(app):
+def copy_binder_files(app, exception):
+    """Copy all Binder requirements and notebooks files."""
+    if exception is not None:
+        return
+
+    if app.builder.name not in ['html', 'readthedocs']:
+        return
+
+    gallery_conf = app.config.sphinx_gallery_conf
+    binder_conf = check_binder_conf(gallery_conf.get('binder'))
+
+    if not len(binder_conf) > 0:
+        return
+
+    logger.info('copying binder requirements...', color='white')
+    _copy_binder_reqs(app, binder_conf)
+    _copy_binder_notebooks(app)
+
+
+def _copy_binder_reqs(app, binder_conf):
     """Copy Binder requirements files to a "binder" folder in the docs."""
-    binder_conf = app.config.sphinx_gallery_conf['binder']
     path_reqs = binder_conf.get('dependencies')
 
-    binder_folder = os.path.join(app.builder.outdir, 'binder')
+    binder_folder = os.path.join(app.outdir, 'binder')
     if not os.path.isdir(binder_folder):
         os.makedirs(binder_folder)
     for path in path_reqs:
-        sh.copy(os.path.join(app.builder.srcdir, path),
-                binder_folder)
+        shutil.copy(os.path.join(app.srcdir, path), binder_folder)
+
+
+def _remove_ipynb_files(path, contents):
+    """Given a list of files in `contents`, remove all files named `ipynb` or
+    directories named `images` and return the result.
+
+    Used with the `shutil` "ignore" keyword to filter out non-ipynb files."""
+    contents_return = []
+    for entry in contents:
+        if entry.endswith('.ipynb'):
+            # Don't include ipynb files
+            pass
+        elif (entry != "images") and os.path.isdir(os.path.join(path, entry)):
+            # Don't include folders not called "images"
+            pass
+        else:
+            # Keep everything else
+            contents_return.append(entry)
+    return contents_return
+
+
+def _copy_binder_notebooks(app):
+    """Copy Jupyter notebooks to the binder notebooks directory.
+
+    Copy each output gallery directory structure but only including the
+    Jupyter notebook files."""
+
+    gallery_conf = app.config.sphinx_gallery_conf
+    gallery_dirs = gallery_conf.get('gallery_dirs')
+    binder_conf = gallery_conf.get('binder')
+    notebooks_dir = os.path.join(app.outdir, binder_conf.get('notebooks_dir'))
+    shutil.rmtree(notebooks_dir, ignore_errors=True)
+    os.makedirs(notebooks_dir)
+
+    iterator = sphinx_compatibility.status_iterator(
+        gallery_dirs, 'copying binder notebooks...', length=len(gallery_dirs))
+
+    for i_folder in iterator:
+        shutil.copytree(os.path.join(app.srcdir, i_folder),
+                        os.path.join(notebooks_dir, i_folder),
+                        ignore=_remove_ipynb_files)
 
 
 def check_binder_conf(binder_conf):
@@ -119,7 +198,7 @@ def check_binder_conf(binder_conf):
 
     # Ensure all fields are populated
     req_values = ['url', 'org', 'repo', 'branch', 'dependencies']
-    optional_values = ['filepath_prefix']
+    optional_values = ['filepath_prefix', 'notebooks_dir', 'use_jupyter_lab']
     missing_values = []
     for val in req_values:
         if binder_conf.get(val) is None:
@@ -150,6 +229,8 @@ def check_binder_conf(binder_conf):
         raise ValueError("`dependencies` value should be a list of strings. "
                          "Got type {}.".format(type(path_reqs)))
 
+    binder_conf['notebooks_dir'] = binder_conf.get('notebooks_dir',
+                                                   'notebooks')
     path_reqs_filenames = [os.path.basename(ii) for ii in path_reqs]
     if not any(ii in path_reqs_filenames for ii in required_reqs_files):
         raise ValueError(
diff --git a/sphinx_gallery/docs_resolv.py b/sphinx_gallery/docs_resolv.py
index 5fb2c8f..cdc726b 100644
--- a/sphinx_gallery/docs_resolv.py
+++ b/sphinx_gallery/docs_resolv.py
@@ -117,7 +117,14 @@ def parse_sphinx_docopts(index):
         elif value == 'true':
             value = True
         else:
-            value = int(value)
+            try:
+                value = int(value)
+            except ValueError:
+                # In Sphinx 1.7.5, URL_ROOT is a JavaScript fragment.
+                # Ignoring this entry since URL_ROOT is not used
+                # elsewhere.
+                # https://github.com/sphinx-gallery/sphinx-gallery/issues/382
+                continue
 
         docopts[key] = value
 
diff --git a/sphinx_gallery/gen_gallery.py b/sphinx_gallery/gen_gallery.py
index 0864dba..c857bce 100644
--- a/sphinx_gallery/gen_gallery.py
+++ b/sphinx_gallery/gen_gallery.py
@@ -21,7 +21,7 @@ from .gen_rst import generate_dir_rst, SPHX_GLR_SIG
 from .docs_resolv import embed_code_links
 from .downloads import generate_zipfiles
 from .sorting import NumberOfCodeLinesSortKey
-from .binder import copy_binder_reqs, check_binder_conf
+from .binder import copy_binder_files, check_binder_conf
 
 try:
     FileNotFoundError
@@ -236,12 +236,6 @@ def generate_gallery_rst(app):
             else:
                 logger.info("\t- %s: not run", fname)
 
-    # Copy the requirements files for binder
-    binder_conf = check_binder_conf(gallery_conf.get('binder'))
-    if len(binder_conf) > 0:
-        logger.info("copying binder requirements...")
-        copy_binder_reqs(app)
-
 
 def touch_empty_backreferences(app, what, name, obj, options, lines):
     """Generate empty back-reference example files
@@ -367,6 +361,7 @@ def setup(app):
 
     app.connect('builder-inited', generate_gallery_rst)
 
+    app.connect('build-finished', copy_binder_files)
     app.connect('build-finished', sumarize_failing_examples)
     app.connect('build-finished', embed_code_links)
     metadata = {'parallel_read_safe': True,
diff --git a/sphinx_gallery/gen_rst.py b/sphinx_gallery/gen_rst.py
index 74893eb..11879a0 100644
--- a/sphinx_gallery/gen_rst.py
+++ b/sphinx_gallery/gen_rst.py
@@ -30,6 +30,7 @@ from distutils.version import LooseVersion
 from .utils import replace_py_ipynb
 
 
+
 # Try Python 2 first, otherwise load from Python 3
 try:
     # textwrap indent only exists in python 3
@@ -83,7 +84,7 @@ from .downloads import CODE_DOWNLOAD
 from .py_source_parser import split_code_and_text_blocks
 
 from .notebook import jupyter_notebook, save_notebook
-from .binder import check_binder_conf, copy_binder_reqs, gen_binder_rst
+from .binder import check_binder_conf, gen_binder_rst
 
 try:
     basestring
@@ -385,7 +386,7 @@ def scale_image(in_fname, out_fname, max_width, max_height):
     # width_sc, height_sc = img.size  # necessary if using thumbnail
 
     # insert centered
-    thumb = Image.new('RGB', (max_width, max_height), (255, 255, 255))
+    thumb = Image.new('RGBA', (max_width, max_height), (255, 255, 255, 255))
     pos_insert = ((max_width - width_sc) // 2, (max_height - height_sc) // 2)
     thumb.paste(img, pos_insert)
 
@@ -710,6 +711,7 @@ def generate_file_rst(fname, target_dir, src_dir, gallery_conf):
     time_m, time_s = divmod(time_elapsed, 60)
     example_nb = jupyter_notebook(script_blocks)
     save_notebook(example_nb, replace_py_ipynb(example_file))
+
     with codecs.open(os.path.join(target_dir, base_image_name + '.rst'),
                      mode='w', encoding='utf-8') as f:
         if time_elapsed >= gallery_conf["min_reported_time"]:
@@ -719,7 +721,8 @@ def generate_file_rst(fname, target_dir, src_dir, gallery_conf):
         # Generate a binder URL if specified
         binder_badge_rst = ''
         if len(binder_conf) > 0:
-            binder_badge_rst += gen_binder_rst(fname, binder_conf)
+            binder_badge_rst += gen_binder_rst(example_file, binder_conf,
+                                               gallery_conf)
 
         example_rst += CODE_DOWNLOAD.format(fname,
                                             replace_py_ipynb(fname),


Test patch:
diff --git a/sphinx_gallery/tests/test_binder.py b/sphinx_gallery/tests/test_binder.py
index da3b5ea..efca66b 100644
--- a/sphinx_gallery/tests/test_binder.py
+++ b/sphinx_gallery/tests/test_binder.py
@@ -12,25 +12,33 @@ from copy import deepcopy
 
 import pytest
 
-from sphinx_gallery.binder import (gen_binder_rst, gen_binder_url,
-                                   check_binder_conf)
-from sphinx_gallery.utils import _TempDir
+from sphinx_gallery.binder import gen_binder_url, check_binder_conf
 
 
 def test_binder():
     """Testing binder URL generation and checks."""
-    file_path = 'myfile.py'
-    conf1 = {'url': 'http://test1.com', 'org': 'org',
-             'repo': 'repo', 'branch': 'branch',
-             'dependencies': '../requirements.txt'}
-    url = gen_binder_url(file_path, conf1)
-    assert url == 'http://test1.com/v2/gh/org/repo/branch?filepath=_downloads/myfile.ipynb'
+    file_path = 'blahblah/mydir/myfile.py'
+    conf_base = {'url': 'http://test1.com', 'org': 'org',
+                 'repo': 'repo', 'branch': 'branch',
+                 'dependencies': '../requirements.txt'}
+    conf_base = check_binder_conf(conf_base)
+    gallery_conf_base = {'gallery_dirs': ['mydir'], 'src_dir': 'blahblah'}
+
+    url = gen_binder_url(file_path, conf_base, gallery_conf_base)
+    expected = ('http://test1.com/v2/gh/org/repo/'
+                'branch?filepath=notebooks/mydir/myfile.ipynb')
+    assert url == expected
 
     # Assert filepath prefix is added
     prefix = 'my_prefix/foo'
+    conf1 = deepcopy(conf_base)
     conf1['filepath_prefix'] = prefix
-    url = gen_binder_url(file_path, conf1)
-    assert url == 'http://test1.com/v2/gh/org/repo/branch?filepath={}/_downloads/myfile.ipynb'.format(prefix)
+    url = gen_binder_url(file_path, conf1, gallery_conf_base)
+    expected = ('http://test1.com/v2/gh/org/repo/'
+                'branch?filepath={}/notebooks/'
+                'mydir/myfile.ipynb').format(prefix)
+
+    assert url == expected
     conf1.pop('filepath_prefix')
 
     # URL must have http
@@ -43,6 +51,8 @@ def test_binder():
 
     # Assert missing params
     for key in conf1.keys():
+        if key == 'notebooks_dir':
+            continue
         conf3 = deepcopy(conf1)
         conf3.pop(key)
         with pytest.raises(ValueError) as excinfo:
@@ -56,7 +66,8 @@ def test_binder():
             conf3 = deepcopy(conf1)
             conf3['dependencies'] = ifile
             url = check_binder_conf(conf3)
-        excinfo.match(r"Did not find one of `requirements.txt` or `environment.yml`")
+        excinfo.match(r"Did not find one of `requirements.txt` "
+                      "or `environment.yml`")
 
     with pytest.raises(ValueError) as excinfo:
         conf6 = deepcopy(conf1)
@@ -76,3 +87,20 @@ def test_binder():
         conf7['foo'] = 'blah'
         url = check_binder_conf(conf7)
     excinfo.match(r"Unknown Binder config key")
+
+    # Assert using lab correctly changes URL
+    conf_lab = deepcopy(conf_base)
+    conf_lab['use_jupyter_lab'] = True
+    url = gen_binder_url(file_path, conf_lab, gallery_conf_base)
+    expected = ('http://test1.com/v2/gh/org/repo/'
+                'branch?urlpath=lab/tree/notebooks/mydir/myfile.ipynb')
+    assert url == expected
+
+    # Assert using static folder correctly changes URL
+    conf_static = deepcopy(conf_base)
+    file_path = 'blahblah/mydir/myfolder/myfile.py'
+    conf_static['notebooks_dir'] = 'ntbk_folder'
+    url = gen_binder_url(file_path, conf_static, gallery_conf_base)
+    expected = ('http://test1.com/v2/gh/org/repo/'
+                'branch?filepath=ntbk_folder/mydir/myfolder/myfile.ipynb')
+    assert url == expected
diff --git a/sphinx_gallery/tests/test_docs_resolv.py b/sphinx_gallery/tests/test_docs_resolv.py
index 94d4d80..6f57510 100644
--- a/sphinx_gallery/tests/test_docs_resolv.py
+++ b/sphinx_gallery/tests/test_docs_resolv.py
@@ -67,6 +67,27 @@ def test_parse_sphinx_docopts():
         'SOURCELINK_SUFFIX': '.txt'
     }
 
+    data_sphinx_175 = '''
+    <script type="text/javascript">
+      var DOCUMENTATION_OPTIONS = {
+        URL_ROOT: document.getElementById("documentation_options")\
+                  .getAttribute('data-url_root'),
+        VERSION:     '2.0.2',
+        COLLAPSE_INDEX: false,
+        FILE_SUFFIX: '.html',
+        HAS_SOURCE:  true,
+        SOURCELINK_SUFFIX: '.txt'
+      };
+    </script>
+    '''
+    assert sg.parse_sphinx_docopts(data_sphinx_175) == {
+        'VERSION': '2.0.2',
+        'COLLAPSE_INDEX': False,
+        'FILE_SUFFIX': '.html',
+        'HAS_SOURCE': True,
+        'SOURCELINK_SUFFIX': '.txt'
+    }
+
     with pytest.raises(ValueError):
         sg.parse_sphinx_docopts('empty input')
 
diff --git a/sphinx_gallery/tests/test_gen_gallery.py b/sphinx_gallery/tests/test_gen_gallery.py
index 61407fd..64276f5 100644
--- a/sphinx_gallery/tests/test_gen_gallery.py
+++ b/sphinx_gallery/tests/test_gen_gallery.py
@@ -102,7 +102,8 @@ def test_no_warning_simple_config(config_app):
     cfg = config_app.config
     assert cfg.project == "Sphinx-Gallery <Tests>"
     build_warn = config_app._warning.getvalue()
-    assert build_warn == ''
+    # ignore 1.8.0 dev bug
+    assert build_warn == '' or 'up extension sphinx.domains.math' in build_warn
 
 
 @pytest.mark.conf_file(content="""
@@ -155,7 +156,8 @@ def test_config_backreferences(config_app):
     assert cfg.sphinx_gallery_conf['backreferences_dir'] == os.path.join(
         'gen_modules', 'backreferences')
     build_warn = config_app._warning.getvalue()
-    assert build_warn == ""
+    # ignore 1.8.0 dev bug
+    assert build_warn == '' or 'up extension sphinx.domains.math' in build_warn
 
 
 def test_duplicate_files_warn(config_app):
@@ -167,13 +169,14 @@ def test_duplicate_files_warn(config_app):
 
     # No warning because no overlapping names
     check_duplicate_filenames(files[:-1])
-    warnings = config_app._warning.getvalue()
-    assert warnings == ''
+    build_warn = config_app._warning.getvalue()
+    # ignore 1.8.0 dev bug
+    assert build_warn == '' or 'up extension sphinx.domains.math' in build_warn
 
     # Warning because last file is named the same
     check_duplicate_filenames(files)
-    warnings = config_app._warning.getvalue()
-    assert msg.format(m) in warnings
+    build_warn = config_app._warning.getvalue()
+    assert msg.format(m) in build_warn
 
 
 def _check_order(config_app, key):
@@ -273,3 +276,34 @@ def test_collect_gallery_files(config_app, tmpdir):
         [ap.strpath for ap in abs_paths if re.search(r'.*\.py$', ap.strpath)])
 
     assert collected_files == expected_files
+
+
+@pytest.mark.conf_file(content="""
+import os
+import sphinx_gallery
+extensions = ['sphinx_gallery.gen_gallery']
+# General information about the project.
+project = u'Sphinx-Gallery <Tests>'
+
+sphinx_gallery_conf = {
+    'backreferences_dir' : os.path.join('modules', 'gen'),
+    'examples_dirs': 'src',
+    'gallery_dirs': ['ex'],
+    'binder': {'url': 'http://test1.com', 'org': 'org',
+               'repo': 'repo', 'branch': 'branch',
+               'notebooks_dir': 'ntbk_folder',
+               'dependencies': 'requirements.txt'}
+}""")
+def test_binder_copy_files(config_app, tmpdir):
+    """Test that notebooks are copied properly."""
+    from sphinx_gallery.binder import copy_binder_files, check_binder_conf
+    gallery_conf = config_app.config.sphinx_gallery_conf
+    # Create requirements file
+    with open(os.path.join(config_app.srcdir, 'requirements.txt'), 'w') as ff:
+        pass
+    copy_binder_files(config_app, None)
+
+    for i_file in ['plot_1', 'plot_2', 'plot_3']:
+        assert os.path.exists(os.path.join(
+            config_app.outdir, 'ntbk_folder', gallery_conf['gallery_dirs'][0],
+            i_file+'.ipynb'))