Repository: sphinx-gallery/sphinx-gallery

Issue:
MAINT: Better error messages
As [pointed out](https://github.com/sphinx-gallery/sphinx-gallery/pull/599/files#r369287580) by @mgeier, raising plain Python errors is not as nice as raising Sphinx-specific variants. We should update our internal exceptions.

Discussion:
codecov-commenter: # [Codecov](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690?src=pr&el=h1) Report
> Merging [#690](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690?src=pr&el=desc) into [master](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/commit/c60e7fcdc40c89ee79ac60f05367539515211d66&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `90.80%`.

[![Impacted file tree graph](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/graphs/tree.svg?width=650&height=150&src=pr&token=K4eytvhwU5)](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #690      +/-   ##
==========================================
+ Coverage   97.81%   97.82%   +0.01%     
==========================================
  Files          32       32              
  Lines        3429     3447      +18     
==========================================
+ Hits         3354     3372      +18     
  Misses         75       75              
```


| [Impacted Files](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sphinx\_gallery/docs\_resolv.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvZG9jc19yZXNvbHYucHk=) | `89.91% <28.57%> (+0.04%)` | :arrow_up: |
| [sphinx\_gallery/gen\_rst.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvZ2VuX3JzdC5weQ==) | `97.57% <80.00%> (+<0.01%)` | :arrow_up: |
| [sphinx\_gallery/scrapers.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvc2NyYXBlcnMucHk=) | `97.74% <80.00%> (+0.01%)` | :arrow_up: |
| [sphinx\_gallery/binder.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvYmluZGVyLnB5) | `95.00% <87.50%> (+0.05%)` | :arrow_up: |
| [sphinx\_gallery/backreferences.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvYmFja3JlZmVyZW5jZXMucHk=) | `96.31% <100.00%> (+0.01%)` | :arrow_up: |
| [sphinx\_gallery/gen\_gallery.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvZ2VuX2dhbGxlcnkucHk=) | `96.95% <100.00%> (+<0.01%)` | :arrow_up: |
| [sphinx\_gallery/py\_source\_parser.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvcHlfc291cmNlX3BhcnNlci5weQ==) | `98.87% <100.00%> (+0.01%)` | :arrow_up: |
| [sphinx\_gallery/sorting.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvc29ydGluZy5weQ==) | `100.00% <100.00%> (ø)` | |
| [sphinx\_gallery/tests/conftest.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvdGVzdHMvY29uZnRlc3QucHk=) | `96.87% <100.00%> (+0.02%)` | :arrow_up: |
| [sphinx\_gallery/tests/test\_backreferences.py](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree#diff-c3BoaW54X2dhbGxlcnkvdGVzdHMvdGVzdF9iYWNrcmVmZXJlbmNlcy5weQ==) | `100.00% <100.00%> (ø)` | |
| ... and [9 more](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690?src=pr&el=footer). Last update [c60e7fc...5c6d804](https://codecov.io/gh/sphinx-gallery/sphinx-gallery/pull/690?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

larsoner: @lucyleeow this one should be ready now
choldgraf: I had no idea that these error classes existed lol. Where did you find them in the sphinx docs?

I'm curious how this changes the behavior of `SG`...does it make the error messages more helpful or something? (+1 on the PR though)
larsoner: > I had no idea that these error classes existed lol. Where did you find them in the sphinx docs?

I really looked here:

https://www.sphinx-doc.org/en/master/_modules/sphinx/errors.html

But I guess the official docs are here:

https://www.sphinx-doc.org/en/master/extdev/appapi.html#exceptions

> I'm curious how this changes the behavior of SG...does it make the error messages more helpful or something?

The output is a bit cleaner. This is what a `ValueError` looks like:
```
Exception occurred:
  File "/home/larsoner/python/sphinx-gallery/sphinx_gallery/gen_rst.py", line 323, in _get_readme
    raise ValueError(
ValueError: Example directory /home/larsoner/python/sphinx-gallery/sphinx_gallery/tests/tinybuild/examplesy/ does not have a README file with one of the expected file extensions ['.txt', '.rst']. Please write one to introduce your gallery.
The full traceback has been saved in /tmp/sphinx-err-bvw0jkjo.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
```
And this is the new one:
```
Extension error:
Example directory /home/larsoner/python/sphinx-gallery/sphinx_gallery/tests/tinybuild/examplesy/ does not have a README file with one of the expected file extensions ['.txt', '.rst']. Please write one to introduce your gallery.
```
choldgraf: ooh I love it, I should re-use those

Patch:
diff --git a/sphinx_gallery/backreferences.py b/sphinx_gallery/backreferences.py
index 688bbbc..e29795c 100644
--- a/sphinx_gallery/backreferences.py
+++ b/sphinx_gallery/backreferences.py
@@ -18,6 +18,8 @@ import os
 import re
 import warnings
 
+from sphinx.errors import ExtensionError
+
 from . import sphinx_compatibility
 from .scrapers import _find_image_ext
 from .utils import _replace_md5
@@ -264,8 +266,8 @@ def _thumbnail_div(target_dir, src_dir, fname, snippet, title,
                      'sphx_glr_%s_thumb.png' % fname[:-3]))
     if check and not os.path.isfile(thumb):
         # This means we have done something wrong in creating our thumbnail!
-        raise RuntimeError('Could not find internal sphinx-gallery thumbnail '
-                           'file:\n%s' % (thumb,))
+        raise ExtensionError('Could not find internal sphinx-gallery thumbnail'
+                             ' file:\n%s' % (thumb,))
     thumb = os.path.relpath(thumb, src_dir)
     full_dir = os.path.relpath(target_dir, src_dir)
 
diff --git a/sphinx_gallery/binder.py b/sphinx_gallery/binder.py
index c498dc1..a09c882 100644
--- a/sphinx_gallery/binder.py
+++ b/sphinx_gallery/binder.py
@@ -18,6 +18,8 @@ change in the future.
 import shutil
 import os
 
+from sphinx.errors import ConfigError
+
 from .utils import replace_py_ipynb
 from . import sphinx_compatibility
 
@@ -134,9 +136,10 @@ def _copy_binder_reqs(app, binder_conf):
     path_reqs = binder_conf.get('dependencies')
     for path in path_reqs:
         if not os.path.exists(os.path.join(app.srcdir, path)):
-            raise ValueError(("Couldn't find the Binder requirements file: {},"
-                              " did you specify the path correctly?"
-                              .format(path)))
+            raise ConfigError(
+                "Couldn't find the Binder requirements file: {}, "
+                "did you specify the path correctly?"
+                .format(path))
 
     binder_folder = os.path.join(app.outdir, 'binder')
     if not os.path.isdir(binder_folder):
@@ -196,7 +199,7 @@ def check_binder_conf(binder_conf):
     # Grab the configuration and return None if it's not configured
     binder_conf = {} if binder_conf is None else binder_conf
     if not isinstance(binder_conf, dict):
-        raise ValueError('`binder_conf` must be a dictionary or None.')
+        raise ConfigError('`binder_conf` must be a dictionary or None.')
     if len(binder_conf) == 0:
         return binder_conf
 
@@ -209,19 +212,19 @@ def check_binder_conf(binder_conf):
             missing_values.append(val)
 
     if len(missing_values) > 0:
-        raise ValueError('binder_conf is missing values for: {}'.format(
+        raise ConfigError('binder_conf is missing values for: {}'.format(
             missing_values))
 
     for key in binder_conf.keys():
         if key not in (req_values + optional_values):
-            raise ValueError("Unknown Binder config key: {}".format(key))
+            raise ConfigError("Unknown Binder config key: {}".format(key))
 
     # Ensure we have http in the URL
     if not any(binder_conf['binderhub_url'].startswith(ii)
                for ii in ['http://', 'https://']):
-        raise ValueError('did not supply a valid url, '
-                         'gave binderhub_url: {}'
-                         .format(binder_conf['binderhub_url']))
+        raise ConfigError('did not supply a valid url, '
+                          'gave binderhub_url: {}'
+                          .format(binder_conf['binderhub_url']))
 
     # Ensure we have at least one dependency file
     # Need at least one of these three files
@@ -231,14 +234,14 @@ def check_binder_conf(binder_conf):
         path_reqs = [path_reqs]
         binder_conf['dependencies'] = path_reqs
     elif not isinstance(path_reqs, (list, tuple)):
-        raise ValueError("`dependencies` value should be a list of strings. "
-                         "Got type {}.".format(type(path_reqs)))
+        raise ConfigError("`dependencies` value should be a list of strings. "
+                          "Got type {}.".format(type(path_reqs)))
 
     binder_conf['notebooks_dir'] = binder_conf.get('notebooks_dir',
                                                    'notebooks')
     path_reqs_filenames = [os.path.basename(ii) for ii in path_reqs]
     if not any(ii in path_reqs_filenames for ii in required_reqs_files):
-        raise ValueError(
+        raise ConfigError(
             'Did not find one of `requirements.txt` or `environment.yml` '
             'in the "dependencies" section of the binder configuration '
             'for sphinx-gallery. A path to at least one of these files '
diff --git a/sphinx_gallery/docs_resolv.py b/sphinx_gallery/docs_resolv.py
index abce811..e3308d0 100644
--- a/sphinx_gallery/docs_resolv.py
+++ b/sphinx_gallery/docs_resolv.py
@@ -19,6 +19,7 @@ import urllib.request as urllib_request
 import urllib.parse as urllib_parse
 from urllib.error import HTTPError, URLError
 
+from sphinx.errors import ExtensionError
 from sphinx.search import js_index
 
 from . import sphinx_compatibility
@@ -39,7 +40,7 @@ def _get_data(url):
         if encoding == 'gzip':
             data = gzip.GzipFile(fileobj=BytesIO(data)).read()
         elif encoding != 'plain':
-            raise RuntimeError('unknown encoding %r' % (encoding,))
+            raise ExtensionError('unknown encoding %r' % (encoding,))
         data = data.decode('utf-8')
     else:
         with codecs.open(url, mode='r', encoding='utf-8') as fid:
@@ -83,13 +84,16 @@ def parse_sphinx_docopts(index):
 
     pos = index.find('var DOCUMENTATION_OPTIONS')
     if pos < 0:
-        raise ValueError('Documentation options could not be found in index.')
+        raise ExtensionError(
+            'Documentation options could not be found in index.')
     pos = index.find('{', pos)
     if pos < 0:
-        raise ValueError('Documentation options could not be found in index.')
+        raise ExtensionError(
+            'Documentation options could not be found in index.')
     endpos = index.find('};', pos)
     if endpos < 0:
-        raise ValueError('Documentation options could not be found in index.')
+        raise ExtensionError(
+            'Documentation options could not be found in index.')
     block = index[pos + 1:endpos].strip()
     docopts = {}
     for line in block.splitlines():
@@ -140,8 +144,9 @@ class SphinxDocLinkResolver(object):
 
         if doc_url.startswith(('http://', 'https://')):
             if relative:
-                raise ValueError('Relative links are only supported for local '
-                                 'URLs (doc_url cannot be absolute)')
+                raise ExtensionError(
+                    'Relative links are only supported for local '
+                    'URLs (doc_url cannot be absolute)')
             index_url = doc_url + '/'
             searchindex_url = doc_url + '/searchindex.js'
             docopts_url = doc_url + '_static/documentation_options.js'
@@ -155,8 +160,9 @@ class SphinxDocLinkResolver(object):
         if (os.name.lower() == 'nt' and
                 not doc_url.startswith(('http://', 'https://'))):
             if not relative:
-                raise ValueError('You have to use relative=True for the local'
-                                 ' package on a Windows system.')
+                raise ExtensionError(
+                    'You have to use relative=True for the local'
+                    ' package on a Windows system.')
             self._is_windows = True
         else:
             self._is_windows = False
diff --git a/sphinx_gallery/gen_gallery.py b/sphinx_gallery/gen_gallery.py
index 9eaf2e6..572620a 100644
--- a/sphinx_gallery/gen_gallery.py
+++ b/sphinx_gallery/gen_gallery.py
@@ -20,6 +20,7 @@ import os
 import pathlib
 from xml.sax.saxutils import quoteattr, escape
 
+from sphinx.errors import ConfigError, ExtensionError
 from sphinx.util.console import red
 from . import sphinx_compatibility, glr_path_static, __version__ as _sg_version
 from .utils import _replace_md5, _has_optipng
@@ -175,11 +176,11 @@ def _complete_gallery_conf(sphinx_gallery_conf, src_dir, plot_gallery,
                     scraper = getattr(scraper, '_get_sg_image_scraper')
                     scraper = scraper()
                 except Exception as exp:
-                    raise ValueError('Unknown image scraper %r, got:\n%s'
-                                     % (orig_scraper, exp))
+                    raise ConfigError('Unknown image scraper %r, got:\n%s'
+                                      % (orig_scraper, exp))
             scrapers[si] = scraper
         if not callable(scraper):
-            raise ValueError('Scraper %r was not callable' % (scraper,))
+            raise ConfigError('Scraper %r was not callable' % (scraper,))
     gallery_conf['image_scrapers'] = tuple(scrapers)
     del scrapers
     # Here we try to set up matplotlib but don't raise an error,
@@ -201,8 +202,8 @@ def _complete_gallery_conf(sphinx_gallery_conf, src_dir, plot_gallery,
     if isinstance(compress_images, str):
         compress_images = [compress_images]
     elif not isinstance(compress_images, (tuple, list)):
-        raise TypeError('compress_images must be a tuple, list, or str, got %s'
-                        % (type(compress_images),))
+        raise ConfigError('compress_images must be a tuple, list, or str, '
+                          'got %s' % (type(compress_images),))
     compress_images = list(compress_images)
     allowed_values = ('images', 'thumbnails')
     pops = list()
@@ -211,9 +212,9 @@ def _complete_gallery_conf(sphinx_gallery_conf, src_dir, plot_gallery,
             if kind.startswith('-'):
                 pops.append(ki)
                 continue
-            raise TypeError('All entries in compress_images must be one of %s '
-                            'or a command-line switch starting with "-", '
-                            'got %r' % (allowed_values, kind))
+            raise ConfigError('All entries in compress_images must be one of '
+                              '%s or a command-line switch starting with "-", '
+                              'got %r' % (allowed_values, kind))
     compress_images_args = [compress_images.pop(p) for p in pops[::-1]]
     if len(compress_images) and not _has_optipng():
         logger.warning(
@@ -231,12 +232,12 @@ def _complete_gallery_conf(sphinx_gallery_conf, src_dir, plot_gallery,
     for ri, resetter in enumerate(resetters):
         if isinstance(resetter, str):
             if resetter not in _reset_dict:
-                raise ValueError('Unknown module resetter named %r'
-                                 % (resetter,))
+                raise ConfigError('Unknown module resetter named %r'
+                                  % (resetter,))
             resetters[ri] = _reset_dict[resetter]
         elif not callable(resetter):
-            raise ValueError('Module resetter %r was not callable'
-                             % (resetter,))
+            raise ConfigError('Module resetter %r was not callable'
+                              % (resetter,))
     gallery_conf['reset_modules'] = tuple(resetters)
 
     lang = lang if lang in ('python', 'python3', 'default') else 'python'
@@ -246,13 +247,13 @@ def _complete_gallery_conf(sphinx_gallery_conf, src_dir, plot_gallery,
     # Ensure the first cell text is a string if we have it
     first_cell = gallery_conf.get("first_notebook_cell")
     if (not isinstance(first_cell, str)) and (first_cell is not None):
-        raise ValueError("The 'first_notebook_cell' parameter must be type "
-                         "str or None, found type %s" % type(first_cell))
+        raise ConfigError("The 'first_notebook_cell' parameter must be type "
+                          "str or None, found type %s" % type(first_cell))
     # Ensure the last cell text is a string if we have it
     last_cell = gallery_conf.get("last_notebook_cell")
     if (not isinstance(last_cell, str)) and (last_cell is not None):
-        raise ValueError("The 'last_notebook_cell' parameter must be type str "
-                         "or None, found type %s" % type(last_cell))
+        raise ConfigError("The 'last_notebook_cell' parameter must be type str"
+                          " or None, found type %s" % type(last_cell))
     # Make it easy to know which builder we're in
     gallery_conf['builder_name'] = builder_name
     gallery_conf['titles'] = {}
@@ -260,21 +261,21 @@ def _complete_gallery_conf(sphinx_gallery_conf, src_dir, plot_gallery,
     backref = gallery_conf['backreferences_dir']
     if (not isinstance(backref, (str, pathlib.Path))) and \
             (backref is not None):
-        raise ValueError("The 'backreferences_dir' parameter must be of type "
-                         "str, pathlib.Path or None, "
-                         "found type %s" % type(backref))
+        raise ConfigError("The 'backreferences_dir' parameter must be of type "
+                          "str, pathlib.Path or None, "
+                          "found type %s" % type(backref))
     # if 'backreferences_dir' is pathlib.Path, make str for Python <=3.5
     # compatibility
     if isinstance(backref, pathlib.Path):
         gallery_conf['backreferences_dir'] = str(backref)
 
     if not isinstance(gallery_conf['css'], (list, tuple)):
-        raise TypeError('gallery_conf["css"] must be list or tuple, got %r'
-                        % (gallery_conf['css'],))
+        raise ConfigError('gallery_conf["css"] must be list or tuple, got %r'
+                          % (gallery_conf['css'],))
     for css in gallery_conf['css']:
         if css not in _KNOWN_CSS:
-            raise ValueError('Unknown css %r, must be one of %r'
-                             % (css, _KNOWN_CSS))
+            raise ConfigError('Unknown css %r, must be one of %r'
+                              % (css, _KNOWN_CSS))
         if gallery_conf['app'] is not None:  # can be None in testing
             gallery_conf['app'].add_css_file(css + '.css')
 
@@ -636,9 +637,10 @@ def summarize_failing_examples(app, exception):
                 color='brown')
 
     if fail_msgs:
-        raise ValueError("Here is a summary of the problems encountered when "
-                         "running the examples\n\n" + "\n".join(fail_msgs) +
-                         "\n" + "-" * 79)
+        raise ExtensionError(
+            "Here is a summary of the problems encountered "
+            "when running the examples\n\n" + "\n".join(fail_msgs) +
+            "\n" + "-" * 79)
 
 
 def collect_gallery_files(examples_dirs, gallery_conf):
diff --git a/sphinx_gallery/gen_rst.py b/sphinx_gallery/gen_rst.py
index 38beee4..ffda8d1 100644
--- a/sphinx_gallery/gen_rst.py
+++ b/sphinx_gallery/gen_rst.py
@@ -33,6 +33,8 @@ import sys
 import traceback
 import codeop
 
+from sphinx.errors import ExtensionError
+
 from .scrapers import (save_figures, ImagePathIterator, clean_modules,
                        _find_image_ext)
 from .utils import (replace_py_ipynb, scale_image, get_md5sum, _replace_md5,
@@ -212,7 +214,7 @@ def extract_intro_and_title(filename, docstring):
     paragraphs = [p for p in paragraphs
                   if not p.startswith('.. ') and len(p) > 0]
     if len(paragraphs) == 0:
-        raise ValueError(
+        raise ExtensionError(
             "Example docstring should have a header for the example title. "
             "Please check the example file:\n {}\n".format(filename))
     # Title is the first paragraph with any ReSTructuredText title chars
@@ -224,7 +226,7 @@ def extract_intro_and_title(filename, docstring):
                       re.MULTILINE)
 
     if match is None:
-        raise ValueError(
+        raise ExtensionError(
             'Could not find a title in first paragraph:\n{}'.format(
                 title_paragraph))
     title = match.group(0).strip()
@@ -281,7 +283,7 @@ def save_thumbnail(image_path_template, src_file, file_conf, gallery_conf):
         image_path = os.path.join(gallery_conf['src_dir'], thumbnail_path)
     else:
         if not isinstance(thumbnail_number, int):
-            raise TypeError(
+            raise ExtensionError(
                 'sphinx_gallery_thumbnail_number setting is not a number, '
                 'got %r' % (thumbnail_number,))
         image_path = image_path_template.format(thumbnail_number)
@@ -318,7 +320,7 @@ def _get_readme(dir_, gallery_conf, raise_error=True):
             if os.path.isfile(fpth):
                 return fpth
     if raise_error:
-        raise FileNotFoundError(
+        raise ExtensionError(
             "Example directory {0} does not have a README file with one "
             "of the expected file extensions {1}. Please write one to "
             "introduce your gallery.".format(dir_, extensions))
diff --git a/sphinx_gallery/py_source_parser.py b/sphinx_gallery/py_source_parser.py
index 5241723..f5faa53 100644
--- a/sphinx_gallery/py_source_parser.py
+++ b/sphinx_gallery/py_source_parser.py
@@ -17,6 +17,7 @@ import sys
 import tokenize
 from textwrap import dedent
 
+from sphinx.errors import ExtensionError
 from .sphinx_compatibility import getLogger
 
 logger = getLogger('sphinx-gallery')
@@ -90,14 +91,16 @@ def _get_docstring_and_rest(filename):
         return SYNTAX_ERROR_DOCSTRING, content, 1, node
 
     if not isinstance(node, ast.Module):
-        raise TypeError("This function only supports modules. "
-                        "You provided {0}".format(node.__class__.__name__))
+        raise ExtensionError("This function only supports modules. "
+                             "You provided {0}"
+                             .format(node.__class__.__name__))
     if not (node.body and isinstance(node.body[0], ast.Expr) and
             isinstance(node.body[0].value, ast.Str)):
-        raise ValueError(('Could not find docstring in file "{0}". '
-                          'A docstring is required by sphinx-gallery '
-                          'unless the file is ignored by "ignore_pattern"')
-                         .format(filename))
+        raise ExtensionError(
+            'Could not find docstring in file "{0}". '
+            'A docstring is required by sphinx-gallery '
+            'unless the file is ignored by "ignore_pattern"'
+            .format(filename))
 
     if LooseVersion(sys.version) >= LooseVersion('3.7'):
         docstring = ast.get_docstring(node)
diff --git a/sphinx_gallery/scrapers.py b/sphinx_gallery/scrapers.py
index d7a7349..73da68a 100644
--- a/sphinx_gallery/scrapers.py
+++ b/sphinx_gallery/scrapers.py
@@ -16,6 +16,7 @@ import sys
 import re
 from textwrap import indent
 
+from sphinx.errors import ExtensionError
 from .utils import scale_image, optipng
 
 __all__ = ['save_figures', 'figure_rst', 'ImagePathIterator', 'clean_modules',
@@ -34,7 +35,7 @@ def _import_matplotlib():
     matplotlib_backend = matplotlib.get_backend().lower()
 
     if matplotlib_backend != 'agg':
-        raise ValueError(
+        raise ExtensionError(
             "Sphinx-Gallery relies on the matplotlib 'agg' backend to "
             "render figures and write them to files. You are "
             "currently using the {} backend. Sphinx-Gallery will "
@@ -264,7 +265,7 @@ class ImagePathIterator(object):
         for ii in range(self._stop):
             yield self.next()
         else:
-            raise RuntimeError('Generated over %s images' % (self._stop,))
+            raise ExtensionError('Generated over %s images' % (self._stop,))
 
     def next(self):
         return self.__next__()
@@ -315,16 +316,17 @@ def save_figures(block, block_vars, gallery_conf):
     for scraper in gallery_conf['image_scrapers']:
         rst = scraper(block, block_vars, gallery_conf)
         if not isinstance(rst, str):
-            raise TypeError('rst from scraper %r was not a string, '
-                            'got type %s:\n%r'
-                            % (scraper, type(rst), rst))
+            raise ExtensionError('rst from scraper %r was not a string, '
+                                 'got type %s:\n%r'
+                                 % (scraper, type(rst), rst))
         n_new = len(image_path_iterator) - prev_count
         for ii in range(n_new):
             current_path, _ = _find_image_ext(
                 image_path_iterator.paths[prev_count + ii])
             if not os.path.isfile(current_path):
-                raise RuntimeError('Scraper %s did not produce expected image:'
-                                   '\n%s' % (scraper, current_path))
+                raise ExtensionError(
+                    'Scraper %s did not produce expected image:'
+                    '\n%s' % (scraper, current_path))
         all_rst += rst
     return all_rst
 
diff --git a/sphinx_gallery/sorting.py b/sphinx_gallery/sorting.py
index 0e29e90..f6df673 100644
--- a/sphinx_gallery/sorting.py
+++ b/sphinx_gallery/sorting.py
@@ -13,6 +13,8 @@ from __future__ import division, absolute_import, print_function
 import os
 import types
 
+from sphinx.errors import ConfigError
+
 from .gen_rst import extract_intro_and_title
 from .py_source_parser import split_code_and_text_blocks
 
@@ -35,9 +37,9 @@ class ExplicitOrder(object):
 
     def __init__(self, ordered_list):
         if not isinstance(ordered_list, (list, tuple, types.GeneratorType)):
-            raise ValueError("ExplicitOrder sorting key takes a list, "
-                             "tuple or Generator, which hold"
-                             "the paths of each gallery subfolder")
+            raise ConfigError("ExplicitOrder sorting key takes a list, "
+                              "tuple or Generator, which hold"
+                              "the paths of each gallery subfolder")
 
         self.ordered_list = list(os.path.normpath(path)
                                  for path in ordered_list)
@@ -46,9 +48,9 @@ class ExplicitOrder(object):
         if item in self.ordered_list:
             return self.ordered_list.index(item)
         else:
-            raise ValueError('If you use an explicit folder ordering, you '
-                             'must specify all folders. Explicit order not '
-                             'found for {}'.format(item))
+            raise ConfigError('If you use an explicit folder ordering, you '
+                              'must specify all folders. Explicit order not '
+                              'found for {}'.format(item))
 
     def __repr__(self):
         return '<%s : %s>' % (self.__class__.__name__, self.ordered_list)
diff --git a/sphinx_gallery/utils.py b/sphinx_gallery/utils.py
index 3a32d34..fa78b88 100644
--- a/sphinx_gallery/utils.py
+++ b/sphinx_gallery/utils.py
@@ -15,6 +15,8 @@ import os
 from shutil import move, copyfile
 import subprocess
 
+from sphinx.errors import ExtensionError
+
 
 def _get_image():
     try:
@@ -23,9 +25,9 @@ def _get_image():
         try:
             import Image
         except ImportError:
-            raise RuntimeError('Could not import pillow, which is required '
-                               'to rescale images (e.g., for thumbnails): %s'
-                               % (exc,))
+            raise ExtensionError(
+                'Could not import pillow, which is required '
+                'to rescale images (e.g., for thumbnails): %s' % (exc,))
     return Image
 
 


Test patch:
diff --git a/sphinx_gallery/tests/conftest.py b/sphinx_gallery/tests/conftest.py
index b5268a1..a78c861 100644
--- a/sphinx_gallery/tests/conftest.py
+++ b/sphinx_gallery/tests/conftest.py
@@ -14,6 +14,7 @@ import pytest
 
 import sphinx
 from sphinx.application import Sphinx
+from sphinx.errors import ExtensionError
 from sphinx.util.docutils import docutils_namespace
 from sphinx_gallery import (docs_resolv, gen_gallery, gen_rst, utils,
                             sphinx_compatibility, py_source_parser)
@@ -156,7 +157,7 @@ def req_mpl():
 def req_pil():
     try:
         _get_image()
-    except RuntimeError:
+    except ExtensionError:
         pytest.skip('Test requires pillow')
 
 
diff --git a/sphinx_gallery/tests/test_backreferences.py b/sphinx_gallery/tests/test_backreferences.py
index b0d25e8..0306986 100644
--- a/sphinx_gallery/tests/test_backreferences.py
+++ b/sphinx_gallery/tests/test_backreferences.py
@@ -7,6 +7,7 @@ Testing the rst files generator
 from __future__ import division, absolute_import, print_function
 
 import pytest
+from sphinx.errors import ExtensionError
 import sphinx_gallery.backreferences as sg
 from sphinx_gallery.py_source_parser import split_code_and_text_blocks
 from sphinx_gallery.gen_rst import _sanitize_rst
@@ -45,7 +46,7 @@ REFERENCE = r"""
 ])
 def test_thumbnail_div(content, tooltip, is_backref):
     """Test if the thumbnail div generates the correct string."""
-    with pytest.raises(RuntimeError, match='internal sphinx-gallery thumb'):
+    with pytest.raises(ExtensionError, match='internal sphinx-gallery thumb'):
         html_div = sg._thumbnail_div('fake_dir', '', 'test_file.py',
                                      '<"test">', '<"title">')
     content = _sanitize_rst(content)
diff --git a/sphinx_gallery/tests/test_binder.py b/sphinx_gallery/tests/test_binder.py
index 1023cc9..fba39f4 100644
--- a/sphinx_gallery/tests/test_binder.py
+++ b/sphinx_gallery/tests/test_binder.py
@@ -10,6 +10,7 @@ from copy import deepcopy
 
 import pytest
 
+from sphinx.errors import ConfigError
 from sphinx_gallery.binder import (gen_binder_url, check_binder_conf,
                                    _copy_binder_reqs)
 
@@ -43,7 +44,7 @@ def test_binder():
     # URL must have http
     conf2 = deepcopy(conf1)
     conf2['binderhub_url'] = 'test1.com'
-    with pytest.raises(ValueError, match='did not supply a valid url'):
+    with pytest.raises(ConfigError, match='did not supply a valid url'):
         url = check_binder_conf(conf2)
 
     # Assert missing params
@@ -52,7 +53,7 @@ def test_binder():
             continue
         conf3 = deepcopy(conf1)
         conf3.pop(key)
-        with pytest.raises(ValueError, match='binder_conf is missing values'):
+        with pytest.raises(ConfigError, match='binder_conf is missing values'):
             url = check_binder_conf(conf3)
 
     # Dependencies file
@@ -60,14 +61,14 @@ def test_binder():
     for ifile in dependency_file_tests:
         conf3 = deepcopy(conf1)
         conf3['dependencies'] = ifile
-        with pytest.raises(ValueError,
+        with pytest.raises(ConfigError,
                            match=r"Did not find one of `requirements.txt` "
                            "or `environment.yml`"):
             url = check_binder_conf(conf3)
 
     conf6 = deepcopy(conf1)
     conf6['dependencies'] = {'test': 'test'}
-    with pytest.raises(ValueError, match='`dependencies` value should be a '
+    with pytest.raises(ConfigError, match='`dependencies` value should be a '
                        'list of strings'):
         url = check_binder_conf(conf6)
 
@@ -80,7 +81,7 @@ def test_binder():
     def apptmp():
         pass
     apptmp.srcdir = '/'
-    with pytest.raises(ValueError, match="Couldn't find the Binder "
+    with pytest.raises(ConfigError, match="Couldn't find the Binder "
                        "requirements file"):
         url = _copy_binder_reqs(apptmp, conf7)
 
@@ -93,7 +94,7 @@ def test_binder():
     # Assert extra unkonwn params
     conf7 = deepcopy(conf1)
     conf7['foo'] = 'blah'
-    with pytest.raises(ValueError, match='Unknown Binder config key'):
+    with pytest.raises(ConfigError, match='Unknown Binder config key'):
         url = check_binder_conf(conf7)
 
     # Assert using lab correctly changes URL
diff --git a/sphinx_gallery/tests/test_docs_resolv.py b/sphinx_gallery/tests/test_docs_resolv.py
index 67334e3..457010e 100644
--- a/sphinx_gallery/tests/test_docs_resolv.py
+++ b/sphinx_gallery/tests/test_docs_resolv.py
@@ -11,6 +11,7 @@ import sys
 
 import pytest
 
+from sphinx.errors import ExtensionError
 import sphinx_gallery.docs_resolv as sg
 
 
@@ -86,11 +87,11 @@ def test_parse_sphinx_docopts():
         'SOURCELINK_SUFFIX': '.txt'
     }
 
-    with pytest.raises(ValueError):
+    with pytest.raises(ExtensionError):
         sg.parse_sphinx_docopts('empty input')
 
-    with pytest.raises(ValueError):
+    with pytest.raises(ExtensionError):
         sg.parse_sphinx_docopts('DOCUMENTATION_OPTIONS = ')
 
-    with pytest.raises(ValueError):
+    with pytest.raises(ExtensionError):
         sg.parse_sphinx_docopts('DOCUMENTATION_OPTIONS = {')
diff --git a/sphinx_gallery/tests/test_full.py b/sphinx_gallery/tests/test_full.py
index 7c58185..a36b73c 100644
--- a/sphinx_gallery/tests/test_full.py
+++ b/sphinx_gallery/tests/test_full.py
@@ -19,6 +19,7 @@ import numpy as np
 from numpy.testing import assert_allclose
 
 from sphinx.application import Sphinx
+from sphinx.errors import ExtensionError
 from sphinx.util.docutils import docutils_namespace
 from sphinx_gallery.utils import _get_image, scale_image, _has_optipng
 
@@ -127,7 +128,7 @@ def test_junit(sphinx_app, tmpdir):
                      buildername='html', status=StringIO())
         # need to build within the context manager
         # for automodule and backrefs to work
-        with pytest.raises(ValueError, match='Here is a summary of the '):
+        with pytest.raises(ExtensionError, match='Here is a summary of the '):
             app.build(False, [])
     junit_file = op.join(new_out_dir, 'sphinx-gallery', 'junit-results.xml')
     assert op.isfile(junit_file)
diff --git a/sphinx_gallery/tests/test_gen_gallery.py b/sphinx_gallery/tests/test_gen_gallery.py
index 2e1b529..8c8661e 100644
--- a/sphinx_gallery/tests/test_gen_gallery.py
+++ b/sphinx_gallery/tests/test_gen_gallery.py
@@ -13,7 +13,7 @@ import re
 
 import pytest
 
-from sphinx.errors import ExtensionError
+from sphinx.errors import ConfigError, ExtensionError
 from sphinx_gallery.gen_gallery import (check_duplicate_filenames,
                                         check_spaces_in_filenames,
                                         collect_gallery_files,
@@ -51,12 +51,14 @@ def test_no_warning_simple_config(sphinx_app_wrapper):
     assert build_warn == ''
 
 
+# This changed from passing the ValueError directly to
+# raising "sphinx.errors.ConfigError" with "threw an exception"
 @pytest.mark.parametrize('err_class, err_match', [
-    pytest.param(ValueError, 'Unknown module resetter',
+    pytest.param(ConfigError, 'Unknown module resetter',
                  id='Resetter unknown',
                  marks=pytest.mark.conf_file(
                      content="sphinx_gallery_conf={'reset_modules': ('f',)}")),
-    pytest.param(ValueError, 'Module resetter .* was not callab',
+    pytest.param(ConfigError, 'Module resetter .* was not callab',
                  id='Resetter not callable',
                  marks=pytest.mark.conf_file(
                      content="sphinx_gallery_conf={'reset_modules': (1.,),}")),
@@ -67,10 +69,10 @@ def test_bad_reset(sphinx_app_wrapper, err_class, err_match):
 
 
 @pytest.mark.parametrize('err_class, err_match', [
-    pytest.param(ValueError, 'Unknown css', id='CSS str error',
+    pytest.param(ConfigError, 'Unknown css', id='CSS str error',
                  marks=pytest.mark.conf_file(
                      content="sphinx_gallery_conf={'css': ('foo',)}")),
-    pytest.param(TypeError, 'must be list or tuple', id="CSS type error",
+    pytest.param(ConfigError, 'must be list or tuple', id="CSS type error",
                  marks=pytest.mark.conf_file(
                      content="sphinx_gallery_conf={'css': 1.}")),
 ])
@@ -315,7 +317,7 @@ def test_failing_examples_raise_exception(sphinx_app_wrapper):
     with codecs.open(os.path.join(example_dir, 'plot_3.py'), 'a',
                      encoding='utf-8') as fid:
         fid.write('raise SyntaxError')
-    with pytest.raises(ValueError) as excinfo:
+    with pytest.raises(ExtensionError) as excinfo:
         sphinx_app_wrapper.build_sphinx_app()
     assert "Unexpected failing examples" in str(excinfo.value)
 
@@ -341,7 +343,7 @@ sphinx_gallery_conf = {
     'expected_failing_examples' :['src/plot_2.py'],
 }""")
 def test_examples_not_expected_to_pass(sphinx_app_wrapper):
-    with pytest.raises(ValueError) as excinfo:
+    with pytest.raises(ExtensionError) as excinfo:
         sphinx_app_wrapper.build_sphinx_app()
     assert "expected to fail, but not failing" in str(excinfo.value)
 
@@ -364,7 +366,7 @@ sphinx_gallery_conf = {
 def test_first_notebook_cell_config(sphinx_app_wrapper):
     from sphinx_gallery.gen_gallery import parse_config
     # First cell must be str
-    with pytest.raises(ValueError):
+    with pytest.raises(ConfigError):
         parse_config(sphinx_app_wrapper.create_sphinx_app())
 
 
@@ -375,7 +377,7 @@ sphinx_gallery_conf = {
 def test_last_notebook_cell_config(sphinx_app_wrapper):
     from sphinx_gallery.gen_gallery import parse_config
     # First cell must be str
-    with pytest.raises(ValueError):
+    with pytest.raises(ConfigError):
         parse_config(sphinx_app_wrapper.create_sphinx_app())
 
 
@@ -386,7 +388,7 @@ sphinx_gallery_conf = {
 def test_backreferences_dir_config(sphinx_app_wrapper):
     """Tests 'backreferences_dir' type checking."""
     from sphinx_gallery.gen_gallery import parse_config
-    with pytest.raises(ValueError,
+    with pytest.raises(ConfigError,
                        match="The 'backreferences_dir' parameter must be of"):
         parse_config(sphinx_app_wrapper.create_sphinx_app())
 
diff --git a/sphinx_gallery/tests/test_gen_rst.py b/sphinx_gallery/tests/test_gen_rst.py
index df6519e..3c23545 100644
--- a/sphinx_gallery/tests/test_gen_rst.py
+++ b/sphinx_gallery/tests/test_gen_rst.py
@@ -18,6 +18,7 @@ import codeop
 
 import pytest
 
+from sphinx.errors import ExtensionError
 import sphinx_gallery.gen_rst as sg
 from sphinx_gallery import downloads
 from sphinx_gallery.gen_gallery import generate_dir_rst
@@ -285,9 +286,9 @@ def test_extract_intro_and_title():
     assert intro_paragraph.replace('\n', ' ')[:95] == intro[:95]
 
     # Errors
-    with pytest.raises(ValueError, match='should have a header'):
+    with pytest.raises(ExtensionError, match='should have a header'):
         sg.extract_intro_and_title('<string>', '')  # no title
-    with pytest.raises(ValueError, match='Could not find a title'):
+    with pytest.raises(ExtensionError, match='Could not find a title'):
         sg.extract_intro_and_title('<string>', '=====')  # no real title
 
 
@@ -450,7 +451,7 @@ def test_gen_dir_rst(gallery_conf, fakesphinxapp, ext):
     args = (gallery_conf['src_dir'], gallery_conf['gallery_dir'],
             gallery_conf, [])
     if ext == '.bad':  # not found with correct ext
-        with pytest.raises(FileNotFoundError, match='does not have a README'):
+        with pytest.raises(ExtensionError, match='does not have a README'):
             generate_dir_rst(*args)
     else:
         out = generate_dir_rst(*args)
diff --git a/sphinx_gallery/tests/test_py_source_parser.py b/sphinx_gallery/tests/test_py_source_parser.py
index 53e6cd0..8c3da35 100644
--- a/sphinx_gallery/tests/test_py_source_parser.py
+++ b/sphinx_gallery/tests/test_py_source_parser.py
@@ -12,6 +12,7 @@ from __future__ import division, absolute_import, print_function
 
 import os.path as op
 import pytest
+from sphinx.errors import ExtensionError
 import sphinx_gallery.py_source_parser as sg
 
 
@@ -23,13 +24,13 @@ def test_get_docstring_and_rest(unicode_sample, tmpdir, monkeypatch):
     fname = op.join(str(tmpdir), 'temp')
     with open(fname, 'w') as fid:
         fid.write('print("hello")\n')
-    with pytest.raises(ValueError, match='Could not find docstring'):
+    with pytest.raises(ExtensionError, match='Could not find docstring'):
         sg._get_docstring_and_rest(fname)
     with open(fname, 'w') as fid:
         fid.write('print hello\n')
     assert sg._get_docstring_and_rest(fname)[0] == sg.SYNTAX_ERROR_DOCSTRING
     monkeypatch.setattr(sg, 'parse_source_file', lambda x: ('', None))
-    with pytest.raises(TypeError, match='only supports modules'):
+    with pytest.raises(ExtensionError, match='only supports modules'):
         sg._get_docstring_and_rest('')
 
 
diff --git a/sphinx_gallery/tests/test_scrapers.py b/sphinx_gallery/tests/test_scrapers.py
index f313f76..4c73f0c 100644
--- a/sphinx_gallery/tests/test_scrapers.py
+++ b/sphinx_gallery/tests/test_scrapers.py
@@ -3,6 +3,7 @@ import os
 import pytest
 import numpy as np
 
+from sphinx.errors import ConfigError, ExtensionError
 import sphinx_gallery
 from sphinx_gallery.gen_gallery import _complete_gallery_conf
 from sphinx_gallery.scrapers import (figure_rst, mayavi_scraper, SINGLE_IMAGE,
@@ -135,13 +136,13 @@ def test_custom_scraper(gallery_conf, monkeypatch):
     # without the monkey patch to add sphinx_gallery._get_sg_image_scraper,
     # we should get an error
     gallery_conf.update(image_scrapers=['sphinx_gallery'])
-    with pytest.raises(ValueError,
+    with pytest.raises(ConfigError,
                        match="has no attribute '_get_sg_image_scraper'"):
         _complete_gallery_conf(*complete_args)
 
     # other degenerate conditions
     gallery_conf.update(image_scrapers=['foo'])
-    with pytest.raises(ValueError, match='Unknown image scraper'):
+    with pytest.raises(ConfigError, match='Unknown image scraper'):
         _complete_gallery_conf(*complete_args)
     gallery_conf.update(image_scrapers=[_custom_func])
     fname_template = os.path.join(gallery_conf['gallery_dir'],
@@ -149,22 +150,22 @@ def test_custom_scraper(gallery_conf, monkeypatch):
     image_path_iterator = ImagePathIterator(fname_template)
     block = ('',) * 3
     block_vars = dict(image_path_iterator=image_path_iterator)
-    with pytest.raises(RuntimeError, match='did not produce expected image'):
+    with pytest.raises(ExtensionError, match='did not produce expected image'):
         save_figures(block, block_vars, gallery_conf)
     gallery_conf.update(image_scrapers=[lambda x, y, z: 1.])
-    with pytest.raises(TypeError, match='was not a string'):
+    with pytest.raises(ExtensionError, match='was not a string'):
         save_figures(block, block_vars, gallery_conf)
     # degenerate string interface
     gallery_conf.update(image_scrapers=['sphinx_gallery'])
     with monkeypatch.context() as m:
         m.setattr(sphinx_gallery, '_get_sg_image_scraper', 'foo',
                   raising=False)
-        with pytest.raises(ValueError, match='^Unknown image.*\n.*callable'):
+        with pytest.raises(ConfigError, match='^Unknown image.*\n.*callable'):
             _complete_gallery_conf(*complete_args)
     with monkeypatch.context() as m:
         m.setattr(sphinx_gallery, '_get_sg_image_scraper', lambda: 'foo',
                   raising=False)
-        with pytest.raises(ValueError, match='^Scraper.*was not callable'):
+        with pytest.raises(ConfigError, match='^Scraper.*was not callable'):
             _complete_gallery_conf(*complete_args)
 
 
@@ -212,6 +213,6 @@ def test_iterator():
     """Test ImagePathIterator."""
     ipi = ImagePathIterator('foo{0}')
     ipi._stop = 10
-    with pytest.raises(RuntimeError, match='10 images'):
+    with pytest.raises(ExtensionError, match='10 images'):
         for ii in ipi:
             pass
diff --git a/sphinx_gallery/tests/test_sorting.py b/sphinx_gallery/tests/test_sorting.py
index 523e9e7..94c00d8 100644
--- a/sphinx_gallery/tests/test_sorting.py
+++ b/sphinx_gallery/tests/test_sorting.py
@@ -11,6 +11,7 @@ from __future__ import division, absolute_import, print_function
 import os.path as op
 import pytest
 
+from sphinx.errors import ConfigError
 from sphinx_gallery.sorting import (ExplicitOrder, NumberOfCodeLinesSortKey,
                                     FileNameSortKey, FileSizeSortKey,
                                     ExampleTitleSortKey)
@@ -26,12 +27,12 @@ def test_ExplicitOrder_sorting_key():
     assert sorted_folders == explicit_folders
 
     # Test fails on wrong input
-    with pytest.raises(ValueError) as excinfo:
+    with pytest.raises(ConfigError) as excinfo:
         ExplicitOrder('nope')
     excinfo.match("ExplicitOrder sorting key takes a list")
 
     # Test missing folder
-    with pytest.raises(ValueError) as excinfo:
+    with pytest.raises(ConfigError) as excinfo:
         sorted_folders = sorted(all_folders, key=key)
     excinfo.match('If you use an explicit folder ordering')