Repository: xonsh/xonsh

Issue:
`xonsh.environ.Env().detype()` seems to skip some environment variables otherwise available with string interpolation
## xonfig

<details>

```
+------------------+--------------------------+
| xonsh            | 0.11.0                   |
| Git SHA          | 4af68065                 |
| Commit Date      | Jan 8 20:16:06 2022      |
| Python           | 3.9.9                    |
| PLY              | 3.11                     |
| have readline    | True                     |
| prompt toolkit   | 3.0.23                   |
| shell type       | prompt_toolkit           |
| history backend  | sqlite                   |
| pygments         | 2.11.2                   |
| on posix         | True                     |
| on linux         | False                    |
| on darwin        | True                     |
| on windows       | False                    |
| on cygwin        | False                    |
| on msys2         | False                    |
| is superuser     | False                    |
| default encoding | utf-8                    |
| xonsh encoding   | utf-8                    |
| encoding errors  | surrogateescape          |
| on jupyter       | False                    |
| jupyter kernel   | None                     |
| xontrib 1        | back2dir                 |
| xontrib 2        | fish_completer           |
| xontrib 3        | jedi                     |
| xontrib 4        | readable-traceback       |
| RC file 1        | /Users/nicholas/.xonshrc |
+------------------+--------------------------+
```

</details>

## Current behavior
I am trying to interact with a xonsh shell from inside a pure-python script.

The following code properly represents the full environment variables (fully typed): `xonsh.environ.XSH.env` (xonsh equivalent is `${...}`

However, calling `${...}.detype()` loses many variables that are available 
For example,   `{'XONSH_LOGIN',
  'XONSH_NO_AMALGAMATE', 'XONSH_LOGIN'}` are both present in `${...}.keys()`, but are missing from `${...}.detype().keys()`

It appears that these missing variables are passed into subprocesses using string interpolation.

For example, `bash -c 'echo $XONSH_DEBUG'` returns `0` but `bash -c @('echo $XONSH_DEBUG')` returns nothing.

#### A bit of prior confusion
This is fixed now, the only problem is with variables in ${...} not being directly available to subprocesses (only through string interpolation).

## Expected Behavior
On my laptop, `${...}.detype().keys()` is currently missing 100 keys that are available in `${...}.keys()`

I think it would be more intuitive if `${...}.detype()` included all the keys originally in `${...}`

This would allow subprocesses to access those "missing vars" with a dynamically computed name (so `bash -c f"${var_name}"` would work properly)

## Steps to Reproduce
 ````xsh
test_env = ${...}
expected_vars = set(test_env.keys())
detyped_vars = set(test_env.detype().keys())
missing_vars = list(expected_vars - detyped_vars)
print("Overall missing vars:", len(missing_vars))
still_directly_in_subproc = []
for var in missing_vars:
    directly_in_subprocess=$(env | grep @(var)).strip() != ""
    if directly_in_subprocess:
        still_directly_in_subproc.append(var) 
print("Missing & directly present in subprocess $(env):", len(still_directly_in_subproc))
# NOTE: Everything present in $(env) is also present in ${...}.detype(), so the above should print 0
# 
# ${...}.env() accurately shows up in `$(env)` but the `missing_vars` do not
````

**EDIT**: In, prior version of this issue mistakenly used `environ.Env()` when I should've been using `environ.XSH.env` (Problem now solved).
<details>
    <summary>Prior confusion</summary>
These variables that are "missing" fall into two categories:
1. Those that are handled with string interpolation (not passed directly)
    - For example `bash -c "echo $XONSH_DEBUG"` works, but `bash -c @("echo $XONSH_DEBUG")` does not
    - Likewise `env | grep XONSH_DEBUG` returns nothing
    - So xonsh is handling these with string interpolation, but they aren't being passed directly
2. those that are passed *directly* to subprocess. They show up in `/usr/bin/env | grep var`, but for some reason not in `environ.Env().detyped()`
    - On my system, these include `XONSHRC`, `LS_COLORS`, `BASH_COMPLETIONS` (and 21 others)
    - Ideally, we would have these in `xonsh.environ.Env().detype()` 😄 
</details>

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**


Discussion:

Patch:
diff --git a/news/env_detype_all.rst b/news/env_detype_all.rst
new file mode 100644
index 00000000..9af2c8c2
--- /dev/null
+++ b/news/env_detype_all.rst
@@ -0,0 +1,23 @@
+**Added:**
+
+* Added ``env.detype_all()`` to get all available variables that is possible to detype.
+
+**Changed:**
+
+* <news item>
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* <news item>
+
+**Fixed:**
+
+* <news item>
+
+**Security:**
+
+* <news item>
diff --git a/xonsh/environ.py b/xonsh/environ.py
index bf697850..f67a1cd7 100644
--- a/xonsh/environ.py
+++ b/xonsh/environ.py
@@ -1988,7 +1988,11 @@ class Env(cabc.MutableMapping):
         return detyped.get(key)
 
     def detype(self):
-        """return a dict that can be used as ``os.environ``"""
+        """
+        Returns a dict of detyped variables.
+        Note! If env variable wasn't explicitly set (e.g. the value has default value in ``Xettings``)
+        it will be not in this list.
+        """
         if self._detyped is not None:
             return self._detyped
         ctx = {}
@@ -2007,6 +2011,24 @@ class Env(cabc.MutableMapping):
         self._detyped = ctx
         return ctx
 
+    def detype_all(self):  # __getitem__
+        """Returns a dict of all available detyped env variables."""
+        if self._detyped is not None:
+            return self._detyped
+        ctx = {}
+        for key in self.rawkeys():
+            if not isinstance(key, str):
+                key = str(key)
+            val = self.__getitem__(key)
+            detyper = self.get_detyper(key)
+            if detyper is not None:
+                val = detyper(val)
+            if not isinstance(val, str):
+                continue
+            ctx[key] = val
+        self._detyped = ctx
+        return ctx
+
     def replace_env(self):
         """Replaces the contents of os_environ with a detyped version
         of the xonsh environment.
diff --git a/xonsh/tools.py b/xonsh/tools.py
index d65950e3..cd97b5ff 100644
--- a/xonsh/tools.py
+++ b/xonsh/tools.py
@@ -208,7 +208,7 @@ class EnvPath(cabc.MutableSequence):
                 self._l = args
             else:
                 raise TypeError(
-                    f"EnvPath cannot be initialized with items " f"of type {type(args)}"
+                    f"EnvPath cannot be initialized with items of type {type(args)}: {args!r}"
                 )
 
     def __getitem__(self, item):


Test patch:
diff --git a/tests/test_environ.py b/tests/test_environ.py
index f5beebb3..9bf551c2 100644
--- a/tests/test_environ.py
+++ b/tests/test_environ.py
@@ -75,6 +75,15 @@ def test_env_detype_no_dict():
     assert "YO" not in det
 
 
+def test_env_detype_all():
+    env = Env()
+    env._vars["DEFAULT"] = Var.with_default(1)
+    env._detyped, det = None, env.detype()
+    env._detyped, det_all = None, env.detype_all()
+    assert "DEFAULT" not in det
+    assert "DEFAULT" in det_all
+
+
 def test_histcontrol_none():
     env = Env(HISTCONTROL=None)
     assert isinstance(env["HISTCONTROL"], set)