Repository: xonsh/xonsh

Issue:
Error when using Xonsh pygments lexer (pyghooks.XonshLexer) outside of a console session 
## xonfig

No actual `xonfig` here - using `xonsh` 0.12.6 as a library

## Expected Behavior
I should be able to use the `XonshLexer` to highlight code using Pygments even if I'm not in a Xonsh console session.

## Current Behavior
When I use the Xonsh lexer in [Pelican](https://blog.getpelican.com/), I get a KeyError because the `XSH.env` dictionary hasn't been initialized, and a check within `pyghooks.XonshLexer` that tries to guard against this doesn't work exactly as it should.

### Traceback (if applicable)

The following happens when I try to generate my Pelican static site that uses the `XonshLexer` to highlight Xonsh code:

<details>

```
  | Traceback (most recent call last):
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pelican/generators.py", line 616, in generate_context
  |     article = self.readers.read_file(
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pelican/readers.py", line 573, in read_file
  |     content, reader_metadata = reader.read(path)
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pelican/readers.py", line 337, in read
  |     content = self._md.convert(text)
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/markdown/core.py", line 267, in convert
  |     newRoot = treeprocessor.run(root)
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/markdown/extensions/codehilite.py", line 224, in run
  |     placeholder = self.md.htmlStash.store(code.hilite())
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/markdown/extensions/codehilite.py", line 122, in hilite
  |     return highlight(self.src, lexer, formatter)
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/__init__.py", line 84, in highlight
  |     return format(lex(code, lexer), formatter, outfile)
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/__init__.py", line 63, in format
  |     formatter.format(tokens, realoutfile)
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/formatter.py", line 95, in format
  |     return self.format_unencoded(tokensource, outfile)
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/formatters/html.py", line 879, in format_unencoded
  |     for t, piece in source:
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/formatters/html.py", line 710, in _wrap_div
  |     for tup in inner:
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/formatters/html.py", line 728, in _wrap_pre
  |     for tup in inner:
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/formatters/html.py", line 734, in _wrap_code
  |     for tup in inner:
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/formatters/html.py", line 753, in _format_lines
  |     for ttype, value in tokensource:
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/pygments/lexer.py", line 188, in streamer
  |     for _, t, v in self.get_tokens_unprocessed(text):
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/xonsh/pyghooks.py", line 1747, in get_tokens_unprocessed
  |     cmd_is_autocd = _command_is_autocd(cmd)
  |   File "/Users/eddiepeters/source/website/.venv/lib/python3.9/site-packages/xonsh/pyghooks.py", line 1611, in _command_is_autocd
  |     if not XSH.env.get("AUTO_CD", False):
  | AttributeError: 'NoneType' object has no attribute 'get'
```

Note that `XSH.env.get("AUTO_CD", False)` does not allow for the non-existence of `XSH.env`.
</details>

Fix:

The problem is that the `XSH.env` property sometimes exists but is not yet initialized as a dictionary.

I believe the following line: https://github.com/xonsh/xonsh/blob/52a12aaf858dc58af773da05bac6007fc525f381/xonsh/pyghooks.py#L1647

Just needs to be changed to:

`if getattr(XSH, "env", None) is None:`

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**


Discussion:

Patch:
diff --git a/news/xonshlexer-fix.rst b/news/xonshlexer-fix.rst
new file mode 100644
index 00000000..9207589a
--- /dev/null
+++ b/news/xonshlexer-fix.rst
@@ -0,0 +1,23 @@
+**Added:**
+
+* <news item>
+
+**Changed:**
+
+* <news item>
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* <news item>
+
+**Fixed:**
+
+* Fixed error caused by unintialized Xonsh session env when using Xonsh as a library just for its Pygments lexer plugin.
+
+**Security:**
+
+* <news item>
diff --git a/xonsh/pyghooks.py b/xonsh/pyghooks.py
index 8c18f285..021c0406 100644
--- a/xonsh/pyghooks.py
+++ b/xonsh/pyghooks.py
@@ -1644,7 +1644,7 @@ class XonshLexer(Python3Lexer):
     def __init__(self, *args, **kwargs):
         # If the lexer is loaded as a pygment plugin, we have to mock
         # __xonsh__.env and __xonsh__.commands_cache
-        if not hasattr(XSH, "env"):
+        if getattr(XSH, "env", None) is None:
             XSH.env = {}
             if ON_WINDOWS:
                 pathext = os_environ.get("PATHEXT", [".EXE", ".BAT", ".CMD"])


Test patch:
diff --git a/tests/test_pyghooks.py b/tests/test_pyghooks.py
index f6fc5e5a..48ec4d1b 100644
--- a/tests/test_pyghooks.py
+++ b/tests/test_pyghooks.py
@@ -9,8 +9,10 @@ import pytest
 from xonsh.environ import LsColors
 from xonsh.platform import ON_WINDOWS
 from xonsh.pyghooks import (
+    XSH,
     Color,
     Token,
+    XonshLexer,
     XonshStyle,
     code_by_name,
     color_file,
@@ -388,3 +390,15 @@ def test_register_custom_pygments_style(name, styles, refrules):
     for rule, color in refrules.items():
         assert rule in style.styles
         assert style.styles[rule] == color
+
+
+def test_can_use_xonsh_lexer_without_xession(xession, monkeypatch):
+    # When Xonsh is used as a library and simply for its lexer plugin, the
+    # xession's env can be unset, so test that it can yield tokens without
+    # that env being set.
+    monkeypatch.setattr(xession, "env", None)
+
+    assert XSH.env is None
+    lexer = XonshLexer()
+    assert XSH.env is not None
+    list(lexer.get_tokens_unprocessed("  some text"))