Repository: ipython/ipython

Issue:
Can the IPython directive use warnings for errors?
Pandas would like to have a workflow where

1. the documentation build always completes to the end, even if there are warnings
2. the CI fails if there are any warnings.

For regular sphinx things, this works with a combination of `-W error --keep-going`, which fails the job at the end of the build if there were any warnings.

This doesn't work with exceptions in IPython blocks. Here's a small example

```
Welcome to test's documentation!
================================

.. ipython:: python

   2 + 'a'

The rest of the biuld

.. ipython:: python

   2 + 2
```

And the relevant IPytyhon conf setting is `ipython_warning_is_error = False`.

When I run

```
sphinx-build -W --keep-going "." _build/
```

The entire document is built, but the exit code is `0`, when I'd like it to be `1`.

I *hope* this is just because exceptions in IPython blocks are printed to stdout. On some basic testing, it looks like using `sphinx.util.logging` and `logger.warning` rather than `sys.stdout.write` will satisfy my needs.

Are the IPython devs open to changing from `sys.stdout.write` to `logger.warning` for this?

cc @tacaswell, and idea if this change would break matplotlib at all?


Discussion:
That seem to be reasonable to me. Feel free to send a pull request. 

@LucianaMarques might also want to take a shot at it ? 
@Carreau yes, thank you! :)

@TomAugspurger I would like to contribute to make this work, can we discuss it? Thanks!
Sure, let me know if you have any questions.

I pushed my example to https://github.com/TomAugspurger/sphinx-test and am
building with `sphinx-build -W --keep-going "." _build/`.
Ideally that would exit with a non-zero exit code.

On Wed, Jan 9, 2019 at 9:29 AM Luciana da Costa Marques <
notifications@github.com> wrote:

> @Carreau <https://github.com/Carreau> yes, thank you! :)
>
> @TomAugspurger <https://github.com/TomAugspurger> I would like to
> contribute to make this work, can we discuss it? Thanks!
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/ipython/ipython/issues/11547#issuecomment-452736945>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHImbJvMvDsoQBNDcyqThRoIHmobCNks5vBgrfgaJpZM4ZzkOJ>
> .
>

@TomAugspurger thank you for the explanation, I will take a loot at it and get back here.
@Carreau could you assign this to me, please?
> Could you assign this to me, please?

I don't think I can assign to people that do not have commit rights. Just leaving a comment saying you want to work on it is enough.


Matplotlib also wants to fail CI if there are any errors in the docs.
Which is to say I am :+1: on this change.

Resolving patch:
diff --git a/IPython/sphinxext/ipython_directive.py b/IPython/sphinxext/ipython_directive.py
--- a/IPython/sphinxext/ipython_directive.py
+++ b/IPython/sphinxext/ipython_directive.py
@@ -200,6 +200,7 @@
 # Third-party
 from docutils.parsers.rst import directives
 from docutils.parsers.rst import Directive
+from sphinx.util import logging
 
 # Our own
 from traitlets.config import Config
@@ -557,15 +558,20 @@ def process_input(self, data, input_prompt, lineno):
             filename = self.directive.state.document.current_source
             lineno = self.directive.state.document.current_line
 
+        # Use sphinx logger for warnings
+        logger = logging.getLogger(__name__)
+
         # output any exceptions raised during execution to stdout
         # unless :okexcept: has been specified.
-        if not is_okexcept and (("Traceback" in processed_output) or ("SyntaxError" in processed_output)):
-            s =  "\nException in %s at block ending on line %s\n" % (filename, lineno)
+        if not is_okexcept and (
+            ("Traceback" in processed_output) or ("SyntaxError" in processed_output)
+        ):
+            s = "\n>>>" + ("-" * 73) + "\n"
+            s += "Exception in %s at block ending on line %s\n" % (filename, lineno)
             s += "Specify :okexcept: as an option in the ipython:: block to suppress this message\n"
-            sys.stdout.write('\n\n>>>' + ('-' * 73))
-            sys.stdout.write(s)
-            sys.stdout.write(processed_output)
-            sys.stdout.write('<<<' + ('-' * 73) + '\n\n')
+            s += processed_output + "\n"
+            s += "<<<" + ("-" * 73)
+            logger.warning(s)
             if self.warning_is_error:
                 raise RuntimeError('Non Expected exception in `{}` line {}'.format(filename, lineno))
 
@@ -573,15 +579,15 @@ def process_input(self, data, input_prompt, lineno):
         # unless :okwarning: has been specified.
         if not is_okwarning:
             for w in ws:
-                s =  "\nWarning in %s at block ending on line %s\n" % (filename, lineno)
+                s = "\n>>>" + ("-" * 73) + "\n"
+                s += "Warning in %s at block ending on line %s\n" % (filename, lineno)
                 s += "Specify :okwarning: as an option in the ipython:: block to suppress this message\n"
-                sys.stdout.write('\n\n>>>' + ('-' * 73))
-                sys.stdout.write(s)
-                sys.stdout.write(('-' * 76) + '\n')
-                s=warnings.formatwarning(w.message, w.category,
-                                         w.filename, w.lineno, w.line)
-                sys.stdout.write(s)
-                sys.stdout.write('<<<' + ('-' * 73) + '\n')
+                s += ("-" * 76) + "\n"
+                s += warnings.formatwarning(
+                    w.message, w.category, w.filename, w.lineno, w.line
+                )
+                s += "<<<" + ("-" * 73)
+                logger.warning(s)
                 if self.warning_is_error:
                     raise RuntimeError('Non Expected warning in `{}` line {}'.format(filename, lineno))
 
@@ -1002,6 +1008,9 @@ def run(self):
         lines = ['.. code-block:: ipython', '']
         figures = []
 
+        # Use sphinx logger for warnings
+        logger = logging.getLogger(__name__)
+
         for part in parts:
             block = block_parser(part, rgxin, rgxout, promptin, promptout)
             if len(block):
@@ -1020,7 +1029,7 @@ def run(self):
                 if self.shell.warning_is_error:
                     raise RuntimeError(message)
                 else:
-                    warnings.warn(message)
+                    logger.warning(message)
 
         for figure in figures:
             lines.append('')


Test patch: