Repository: ipython/ipython

Issue:
Sphinx ipython directive greedily strips all decorators, including valid python ones!
A sphinx ipython code cell which contains a valid python decorator will currently have the line with the decorator completely stripped before the code is run!

In my case that meant that this code block

```python
.. ipython:: python
    from xgcm import as_grid_ufunc

    @as_grid_ufunc(signature="(ax1:center)->(ax1:left)")
    def diff_forward(a):
        return a - np.roll(a, -1, axis=-1)
```
got executed as this instead
```python
from xgcm import as_grid_ufunc

def diff_forward(a):
    return a - np.roll(a, -1, axis=-1)
```
(which caused an error for me later on). I even renders the html with the missing code gone

![Screenshot from 2022-02-16 15-00-17](https://user-images.githubusercontent.com/35968931/154346260-e30741cd-31dd-4919-8c60-198993289b81.png)


It's kind of difficult to make a minimum executable example of this bug because normally what happens is the removal of the decorator leads to code that is invalid in some less clear way, such as giving the error 

```
RuntimeError: Code input with no code at /home/tom/Documents/Work/Code/xgcm/doc/grid_ufuncs.rst, line None
```

That's what happened when I tried to make this MWE:

```python

..ipython:: python

    def decorator(func):
        print("Ran the decorator code")
    
    @decorator
    def f():
        print("Ran the undecorated function call")
    
    f()
```

---

Regardless, the problem can be traced back to this line

https://github.com/ipython/ipython/blob/6c44d22a240fc79eba76995c0a04f79cae868f8c/IPython/sphinxext/ipython_directive.py#L266

which implicitly assumes that ANY decorator in the code must be one of the sphinx ipython_directive extension's "pseudo-decorators", and removes it!

This greedy line also causes another bug (/missing feature) of not being able to chain multiple pseudo-decorators together, but the code author (@fperez) seems to have known about that bug.

---

I think the fix for the first bug should be relatively simple, we just need to only strip decorators which actually match those "pseudo-decorators" defined in the ipython_directive's API.






Discussion:
closed by #13532 
This needs to be re-opened, see https://github.com/ipython/ipython/pull/13532#issuecomment-1081174857:

> Okay so this did not fix the original problem @Carreau, so issue https://github.com/ipython/ipython/issues/13531 should be re-opened.
> 
> I did a dev install of ipython and now I think the actual problem is that the regex that identifies a valid input line. If I insert this debugging code
> 
> ```python
> if line_stripped.startswith('@'):
>     print("a decorator of some sort was found")
>     print(rgxin.match(line))
> ```
> within the while loop in `block_parser` then I find that for an input line starting with `@` it prints `"a decorator of some sort was found"` but prints `None` when it's supposed to find a match using that regex. This would mean that lines starting with `@` are never processed as input lines.
> 
> I'm not sure what exactly is wrong with the regex though, and I'm confused as to why in the `ipython_directive` code it will match for example `In[0]: arr` but when I create the same regex in a separate ipython session it doesn't seem to match the same string... :confused:

Resolving patch:
diff --git a/IPython/sphinxext/ipython_directive.py b/IPython/sphinxext/ipython_directive.py
--- a/IPython/sphinxext/ipython_directive.py
+++ b/IPython/sphinxext/ipython_directive.py
@@ -821,8 +821,11 @@ def process_pure_python(self, content):
                 output.append(line)
                 continue
 
-            # handle decorators
-            if line_stripped.startswith('@'):
+            # handle pseudo-decorators, whilst ensuring real python decorators are treated as input
+            if any(
+                line_stripped.startswith("@" + pseudo_decorator)
+                for pseudo_decorator in PSEUDO_DECORATORS
+            ):
                 output.extend([line])
                 if 'savefig' in line:
                     savefig = True # and need to clear figure


Test patch: