Repository: xonsh/xonsh

Issue:
`AttributeError` with `$XONSH_STDERR_PREFIX` when rc file outputs to stderr
First of all, thanks for a great shell!

When `$XONSH_STDERR_PREFIX` (and `$XONSH_STDERR_POSTFIX`) are defined, and a command in the Xonsh rc file outputs to stderr, an `AttributeError: 'NoneType' object has no attribute 'shell'` gets raised instead of showing the stderr output.

## xonfig

<details>
(from a `xonsh` instance running the example rc file from below)

```
$ xonfig
+------------------+----------------------+
| xonsh            | 0.8.12               |
| Git SHA          | 986bf28d             |
| Commit Date      | Mar 11 14:37:52 2019 |
| Python           | 3.6.7                |
| PLY              | 3.11                 |
| have readline    | True                 |
| prompt toolkit   | 2.0.9                |
| shell type       | prompt_toolkit2      |
| pygments         | 2.3.1                |
| on posix         | True                 |
| on linux         | True                 |
| distro           | ubuntu               |
| on darwin        | False                |
| on windows       | False                |
| on cygwin        | False                |
| on msys2         | False                |
| is superuser     | False                |
| default encoding | utf-8                |
| xonsh encoding   | utf-8                |
| encoding errors  | surrogateescape      |
+------------------+----------------------+
```
</details>

## Expected Behavior
Show the actual output that was sent to stderr.  This is what actually happens when `$XONSH_STDERR_PREFIX` (and `$XONSH_STDERR_POSTFIX`) aren't defined.

## Current Behavior
A traceback from an internal Xonsh error is shown.

### Traceback (if applicable)

<details>

```
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/environ.py", line 1642, in xonsh_script_run_control
    run_script_with_cache(filename, execer, ctx)
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/codecache.py", line 162, in run_script_with_cache
    run_compiled_code(ccode, glb, loc, mode)
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/codecache.py", line 67, in run_compiled_code
    func(code, glb, loc)
  File "xonsh_repro", line 4, in <module>
    ls /non_existant
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/built_ins.py", line 987, in subproc_captured_hiddenobject
    return run_subproc(cmds, captured="hiddenobject")
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/built_ins.py", line 950, in run_subproc
    command.end()
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/proc.py", line 2117, in end
    self._end(tee_output=tee_output)
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/proc.py", line 2125, in _end
    for _ in self.tee_stdout():
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/proc.py", line 2038, in tee_stdout
    for line in self.iterraw():
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/proc.py", line 2002, in iterraw
    self.stream_stderr(safe_readlines(stderr))
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/proc.py", line 2065, in stream_stderr
    if self.stderr_prefix:
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/proc.py", line 2384, in stderr_prefix
    s = format_std_prepost(t, env=env)
  File "/home/user/.local/lib/python3.6/site-packages/xonsh/tools.py", line 1945, in format_std_prepost
    shell = builtins.__xonsh__.shell.shell
AttributeError: 'NoneType' object has no attribute 'shell'
error running xonsh run control file '/home/user/xonsh_repro': 'NoneType' object has no attribute 'shell'
```
</details>

## Steps to Reproduce
```#!sh
$  cat xonsh_repro
$XONSH_STDERR_PREFIX = '{RED}'
$XONSH_STDERR_POSTFIX = '{NO_COLOR}'

ls /non_existant
$  xonsh --rc xonsh_repro
```



Discussion:
> First of all, thanks for a great shell!

Thanks for reporting @sevens-ef! And thanks for the kind word.  I am looking into this now.
sorry for the delay
@sevens-ef - I cannot reproduce this on v0.9.5.  Is this still an issue for you?
Hi @scopatz,

> sorry for the delay

No problem.

> @sevens-ef - I cannot reproduce this on v0.9.5. Is this still an issue for you?

I can still reproduce it, with the same reproduction steps as in my original report.  Tested both v0.9.4 and 0.9.5.  For what it's worth, I also tested 0.9.5 with both Python 3.6.7 with prompt toolkit 2, and Python 3.5.3 with prompt toolkit 1, both variants show the same issue.  Also tested it from a Xonsh installed into a new virtualenv (Python 3.6 and prompt toolkit 2, both with and without pygments), could reproduce it there as well.

However, to get the full traceback shown, `$XONSH_SHOW_TRACEBACK = True` needs to be set (which makes sense... :P).  In my earlier tests, this got inherited from the Xonsh session in which I did the tests...  Testing outside of a Xonsh session, without setting `$XONSH_SHOW_TRACEBACK` I don't get the traceback of course, just the error itself; e.g.:

```
xonsh: For full traceback set: $XONSH_SHOW_TRACEBACK = True
AttributeError: 'NoneType' object has no attribute 'shell'
error running xonsh run control file '/home/user/xonsh_repro': 'NoneType' object has no attribute 'shell'
```

Adding `$XONSH_SHOW_TRACEBACK = True` to the testing rc file shows the same traceback as in my original report again.

To be sure, this is the full rc file I used in this run of tests:

```
$XONSH_SHOW_TRACEBACK = True

$XONSH_STDERR_PREFIX = '{RED}'
$XONSH_STDERR_POSTFIX = '{NO_COLOR}'

ls /non_existant
```

Also, using `env -i` to launch the testing Xonsh (should give a fully clean env) also shows the same traceback, though this breaks many other things (as is expected, as even basic things like `$HOME` and `$USER` are now gone).  Command used: `env -i XONSH_DEBUG=1 xonsh --rc ~/xonsh_repro`

Let me know if there's more I can do/test.

Oooooo I see. You are modifying these in the xonshrc.  Yeah, this is weird.  I can reproduce that now.
So this is happening because you are both setting the env vars and running a command in the xonshrc when the shell starts up.  The xonshrc is executed before the Shell class is fully started up because we need to be able to set configuration variables for the shell itself in the xonshrc.  Also, right now the prompt_formatter function is used to do the stderr formatting. 

I wonder if there is a way around this.

Resolving patch:
diff --git a/xonsh/tools.py b/xonsh/tools.py
--- a/xonsh/tools.py
+++ b/xonsh/tools.py
@@ -1948,16 +1948,27 @@ def format_std_prepost(template, env=None):
     if not template:
         return ""
     env = builtins.__xonsh__.env if env is None else env
-    shell = builtins.__xonsh__.shell.shell
-    try:
-        s = shell.prompt_formatter(template)
-    except Exception:
-        print_exception()
-    # \001\002 is there to fool pygments into not returning an empty string
-    # for potentially empty input. This happens when the template is just a
-    # color code with no visible text.
     invis = "\001\002"
-    s = shell.format_color(invis + s + invis, force_string=True)
+    if builtins.__xonsh__.shell is None:
+        # shell hasn't fully started up (probably still in xonshrc)
+        from xonsh.prompt.base import PromptFormatter
+        from xonsh.ansi_colors import ansi_partial_color_format
+
+        pf = PromptFormatter()
+        s = pf(template)
+        style = env.get("XONSH_COLOR_STYLE")
+        s = ansi_partial_color_format(invis + s + invis, hide=False, style=style)
+    else:
+        # shell has fully started. do the normal thing
+        shell = builtins.__xonsh__.shell.shell
+        try:
+            s = shell.prompt_formatter(template)
+        except Exception:
+            print_exception()
+        # \001\002 is there to fool pygments into not returning an empty string
+        # for potentially empty input. This happens when the template is just a
+        # color code with no visible text.
+        s = shell.format_color(invis + s + invis, force_string=True)
     s = s.replace(invis, "")
     return s
 


Test patch: