Repository: jupyter/jupyter-sphinx

Issue:
Allow disabling of stderr printing
Sometimes when importing certain packages there are various warnings that are raised. These warnings are often unavoidable and are not the fault of the end user.

Currently we raise an exception whenever anything is printed to `stderr`, however it seems that the correct way to handle these warning messages is just to suppress them entirely; i.e. not to print to `stderr` and also not to raise an exception.

I propose a configuration parameter that suppresses `stderr` and does not raise an exception when it is printed to.

Discussion:
akhmerov: Thanks for finding the issue and proposing the fix. The implementation looks solid, however the main choice that we have to make is the UI for controlling this behavior.

We have to choose between default hiding, default showing, default raising on stderr.

Further, we should decide at which levels out of conf.py / single kernel / single cell should the user be able to control this behavior.

This decision depends on the typical usage patterns that we anticipate.
akhmerov: Also I think that emitting a sphinx warning if there's stderr is a useful behavior.
akhmerov: Possible usage scenarios, sorted according to my estimation for how common they are, from most to least frequent.

- The user does not want to show stderr but wants to know that the code produces stderr/warnings throughout the complete documentation
- The user expects the code to produce no stderr, considers stderr an error
- The user has a handful cases where they want to actually show stderr
akhmerov: Assuming the above collection of scenarios is correct, it seems that the correct approach is:

- To always emit extension warnings when stderr is produced. The users may then use sphinx tools to either convert these to errors or silence them
- To allow a per-cell option for including stderr, which would then produce no warning
akhmerov: @jbweston what do you think?
jbweston: Yes that seems like the correct thing
akhmerov: What should we call the new option? Is just `:stderr:` clear enough?
jbweston: I think so; it's not a new option; it just does what it did before. AFAICT the only difference now is that there is a warning raised and not an error
akhmerov: Thanks for implementing the changes. Will you have a chance to fix the tests as well?

Patch:
diff --git a/jupyter_sphinx/execute.py b/jupyter_sphinx/execute.py
index 9207f9a..7fc5160 100644
--- a/jupyter_sphinx/execute.py
+++ b/jupyter_sphinx/execute.py
@@ -379,8 +379,8 @@ class ExecuteJupyterCells(SphinxTransform):
                           if output['output_type'] == 'stream'
                              and output['name'] == 'stderr']
                 if stderr and not node.attributes['stderr']:
-                    raise ExtensionError('Cell printed to stderr:\n{}'
-                                         .format(stderr[0]['text']))
+                    logger.warning('Cell printed to stderr:\n{}'
+                                   .format(stderr[0]['text']))
 
             try:
                 lexer = notebook.metadata.language_info.pygments_lexer
@@ -407,6 +407,7 @@ class ExecuteJupyterCells(SphinxTransform):
                 output_nodes = cell_output_to_nodes(
                     cell,
                     self.config.jupyter_execute_data_priority,
+                    bool(node.attributes["stderr"]),
                     sphinx_abs_dir(self.env),
                     thebe_config
                 )
@@ -463,7 +464,7 @@ def split_on(pred, it):
     return (list(x) for _, x in groupby(it, count))
 
 
-def cell_output_to_nodes(cell, data_priority, dir, thebe_config):
+def cell_output_to_nodes(cell, data_priority, write_stderr, dir, thebe_config):
     """Convert a jupyter cell with outputs and filenames to doctree nodes.
 
     Parameters
@@ -471,6 +472,8 @@ def cell_output_to_nodes(cell, data_priority, dir, thebe_config):
     cell : jupyter cell
     data_priority : list of mime types
         Which media types to prioritize.
+    write_stderr : bool
+        If True include stderr in cell output
     dir : string
         Sphinx "absolute path" to the output folder, so it is a relative path
         to the source folder prefixed with ``/``.
@@ -483,6 +486,8 @@ def cell_output_to_nodes(cell, data_priority, dir, thebe_config):
         if (
             output_type == 'stream'
         ):
+            if not write_stderr and output["name"] == "stderr":
+                continue
             to_add.append(docutils.nodes.literal_block(
                 text=output['text'],
                 rawsource=output['text'],


Test patch:
diff --git a/tests/test_execute.py b/tests/test_execute.py
index dea83b6..0c11eb4 100644
--- a/tests/test_execute.py
+++ b/tests/test_execute.py
@@ -26,20 +26,27 @@ def doctree():
     apps = []
     syspath = sys.path[:]
 
-    def doctree(source, config=None):
+    def doctree(source, config=None, return_warnings=False):
         src_dir = tempfile.mkdtemp()
         source_trees.append(src_dir)
         with open(os.path.join(src_dir, 'conf.py'), 'w') as f:
             f.write("extensions = ['jupyter_sphinx.execute']")
             if config is not None:
                 f.write('\n' + config)
-        with open(os.path.join(src_dir, 'index.rst'), 'w') as f:
+        with open(os.path.join(src_dir, 'contents.rst'), 'w') as f:
             f.write(source)
+        warnings = StringIO()
         app = SphinxTestApp(srcdir=path(src_dir), status=StringIO(),
-                            warning=StringIO())
+                            warning=warnings)
         apps.append(app)
         app.build()
-        return app.env.get_doctree('index')
+
+        doctree = app.env.get_doctree("contents")
+
+        if return_warnings:
+            return doctree, warnings.getvalue()
+        else:
+            return doctree
 
     yield doctree
 
@@ -217,8 +224,11 @@ def test_stderr(doctree):
         import sys
         print('hello world', file=sys.stderr)
     """
-    with pytest.raises(ExtensionError):
-        tree = doctree(source)
+
+    tree, warnings = doctree(source, return_warnings=True)
+    assert "hello world" in warnings
+    cell, = tree.traverse(JupyterCellNode)
+    assert len(cell.children) == 1  # no output
 
     source = """
     .. jupyter-execute::