internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: xonsh/xonsh

Issue:
Input Lag with fresh install in WSL 2 with conda installation
<!--- Provide a general summary of the issue in the Title above -->
<!--- If you have a question along the lines of "How do I do this Bash command in xonsh"
please first look over the Bash to Xonsh translation guide: https://xon.sh/bash_to_xsh.html
If you don't find an answer there, please do open an issue! -->

## System information

WSL 2 Debian 10
Windows 10 Pro 2004

## xonfig

<details>

I don't have any xonfig yet, it is a fresh install.

</details>

## Expected Behavior
Not have any lag, like with zsh or bash (or even fish).

## Current Behavior
<!--- Tell us what happens instead of the expected behavior -->
<!--- If part of your bug report is a traceback, please first enter debug mode before triggering the error
To enter debug mode, set the environment variable `XONSH_DEBUG=1` _before_ starting `xonsh`.
On Linux and OSX, an easy way to to do this is to run `env XONSH_DEBUG=1 xonsh` -->
There is input lag with the fresh install of xonsh. A very noticeable input lag.

## Steps to Reproduce
<!--- Please try to write out a minimal reproducible snippet to trigger the bug, it will help us fix it! -->
1. Install the latest version of conda using:

```sh
wget https://repo.anaconda.com/archive/Anaconda3-2020.07-Linux-x86_64.sh
bash Anaconda3-2020.07-Linux-x86_64.sh
```

2. Install xonsh with conda

```sh
conda config --add channels conda-forge
conda install xonsh
```

3. Open xonsh

```sh
xonsh
```

## Additional details

<details>

I tried to use a fresh **OpenSUSE WSL** installation and there is input lag, but if I install and old version with `zipper` there is no input lag.

</details>

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**



Discussion:
Hi @datwaft ! Please check it out - #3895
And please change WSL to WSL2 in the title.
> Hi @datwaft ! Please check it out - #3895
> And please change WSL to WSL2 in the title.

Thank you! I tried to google it but didn't find anything. That explains it.

Resolving patch:
diff --git a/xonsh/environ.py b/xonsh/environ.py
--- a/xonsh/environ.py
+++ b/xonsh/environ.py
@@ -1739,7 +1739,7 @@ def DEFAULT_VARS():
             is_int,
             int,
             str,
-            10000,
+            6000,
             "Number of files on the PATH above which a warning is shown.",
         ),
     }


Test patch:
Repository: xonsh/xonsh

Issue:
TypeError which is causing xonfig wizard to fail
```
xonsh: To log full traceback to a file set: $XONSH_TRACEBACK_LOGFILE = <filename>
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/xonsh/procs/__amalgam__.py", line 2287, in run
    r = self.f(self.args, sp_stdin, sp_stdout, sp_stderr, spec, spec.stack)
  File "/usr/lib/python3.8/site-packages/xonsh/procs/__amalgam__.py", line 2079, in proxy_two
    return f(args, stdin)
  File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 19415, in xonfig
    return xonfig_main(args)
  File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 15007, in xonfig_main
    return XONFIG_MAIN_ACTIONS[ns.action](ns)
  File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 14664, in _wizard
    w = make_xonfig_wizard(
  File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 14627, in make_xonfig_wizard
    question=WIZARD_XONTRIB_QUESTION, yes=make_xontribs_wiz(), no=wiz.Pass()
  File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 14601, in make_xontribs_wiz
    return _make_flat_wiz(make_xontrib, get_xontribs().items())
  File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 14552, in _make_flat_wiz
    for k in kids:
TypeError: make_xontrib() missing 1 required positional argument: 'xontrib'
```



Discussion:
Hi @SolarAquarion - what is the output of `xonfig`?

Resolving patch:
diff --git a/xonsh/xonfig.py b/xonsh/xonfig.py
--- a/xonsh/xonfig.py
+++ b/xonsh/xonfig.py
@@ -357,8 +357,9 @@ def _xontrib_path(visitor=None, node=None, val=None):
     return ("xontribs", len(visitor.state.get("xontribs", ())))
 
 
-def make_xontrib(name: str, xontrib: Xontrib):
+def make_xontrib(xon_item: tp.Tuple[str, Xontrib]):
     """Makes a message and StoreNonEmpty node for a xontrib."""
+    name, xontrib = xon_item
     name = name or "<unknown-xontrib-name>"
     msg = "\n{BOLD_CYAN}" + name + "{RESET}\n"
     if xontrib.url:


Test patch:
Repository: xonsh/xonsh

Issue:
avox loading
It seems that `avox` since 0.8 needs `autovox` to be loaded to work (see https://github.com/AstraLuma/xontrib-avox#configuration, https://github.com/AstraLuma/xontrib-avox/releases/tag/v0.8).

It would be nice to have the docs (in particular https://xon.sh/xontribs.html#avox) updated.

Let's ping @AstraLuma on this.

On the `avox` side, it may be nice to have a warning message displayed if `autovox` is not loaded.


Discussion:

Resolving patch:
diff --git a/xonsh/xontribs_meta.py b/xonsh/xontribs_meta.py
--- a/xonsh/xontribs_meta.py
+++ b/xonsh/xontribs_meta.py
@@ -131,8 +131,7 @@ def define_xontribs():
         ),
         "avox": Xontrib(
             url="https://github.com/AstraLuma/xontrib-avox",
-            description="Automatic (de)activation of virtual environments as "
-            "you cd around",
+            description="Policy for autovox based on project directories",
             package=_XontribPkg(
                 name="xontrib-avox",
                 license="GPLv3",


Test patch:
Repository: xonsh/xonsh

Issue:
Using fastest json library to increase the speed of json history
Hi! I am a snailger, And I ride, and I ride, I ride through the ocean backsides, I see the conchs come out of the sky.

What about using fastest json library to increase the speed of json history?

For example:

* orjson - https://github.com/ijl/orjson - the fastest Python library for JSON.
* ujson - https://pypi.org/project/ujson/ - UltraJSON is an ultra fast JSON encoder and decoder written in pure C with bindings for Python 3.6+.
* And the standard json as fallback.

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**



Discussion:
Load time of 2785 history json files in xonsh:
* json - 0.62s
* ujson - 0.52s
* orjson - 0.50s

Resolving patch:
diff --git a/xonsh/history/json.py b/xonsh/history/json.py
--- a/xonsh/history/json.py
+++ b/xonsh/history/json.py
@@ -3,12 +3,16 @@
 import os
 import sys
 import time
-import json
 import builtins
 import collections
 import threading
 import collections.abc as cabc
 
+try:
+    import ujson as json
+except ImportError:
+    import json  # type: ignore
+
 from xonsh.history.base import History
 import xonsh.tools as xt
 import xonsh.lazyjson as xlj
@@ -180,6 +184,7 @@ def files(self, only_unlocked=False):
         boot = uptime.boottime()
         fs = _xhj_get_history_files(sort=False)
         files = []
+        time_start = time.time()
         for f in fs:
             try:
                 cur_file_size = os.path.getsize(f)
@@ -206,7 +211,12 @@ def files(self, only_unlocked=False):
                 )
                 lj.close()
                 if xonsh_debug:
-                    print(f"... Enumerated {len(files):7d} history files.\r", end="")
+                    time_lag = time.time() - time_start
+                    print(
+                        f"[history.{json.__name__}] Enumerated {len(files):,d} history files for {time_lag:0.4f}s.\r",
+                        end="",
+                        file=sys.stderr,
+                    )
             except (IOError, OSError, ValueError):
                 continue
         files.sort()  # this sorts by elements of the tuple,
@@ -477,7 +487,7 @@ def all_items(self, newest_first=False, **kwargs):
                 continue
             try:
                 commands = json_file.load()["cmds"]
-            except json.decoder.JSONDecodeError:
+            except (json.decoder.JSONDecodeError, ValueError):
                 # file is corrupted somehow
                 if builtins.__xonsh__.env.get("XONSH_DEBUG") > 0:
                     msg = "xonsh history file {0!r} is not valid JSON"
diff --git a/xonsh/lazyjson.py b/xonsh/lazyjson.py
--- a/xonsh/lazyjson.py
+++ b/xonsh/lazyjson.py
@@ -1,11 +1,15 @@
 # -*- coding: utf-8 -*-
 """Implements a lazy JSON file class that wraps around json data."""
 import io
-import json
 import weakref
 import contextlib
 import collections.abc as cabc
 
+try:
+    import ujson as json
+except ImportError:
+    import json  # type: ignore
+
 
 def _to_json_with_size(obj, offset=0, sort_keys=False):
     if isinstance(obj, str):


Test patch:
Repository: xonsh/xonsh

Issue:
XONSH_HISTORY_FILE is not actual
Hi!

I've found that `XONSH_HISTORY_FILE` is not actual:

```python
history info | grep filename
# filename: /home/user/.local/share/xonsh/xonsh-history.sqlite

$XONSH_HISTORY_FILE
# '/home/user/.xonsh_history.json'

# Prepared by xontrib-hist-format

```

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**



Discussion:
Hey @gforsyth you did a clean up in #4145 may be the code above is also outdated?
Hey @anki-code -- yes, I agree we can remove that env-var now.  Should be straightforward, although there are a bunch of sample test files that we _may_ need to edit

Resolving patch:
diff --git a/xonsh/environ.py b/xonsh/environ.py
--- a/xonsh/environ.py
+++ b/xonsh/environ.py
@@ -1397,10 +1397,11 @@ class PromptSetting(Xettings):
         "from ``xonsh.history.base.History``, or its instance.",
     )
     XONSH_HISTORY_FILE = Var.with_default(
-        os.path.expanduser("~/.xonsh_history.json"),
-        "Location of history file (deprecated).",
+        None,
+        "Location of history file that was set by history backend.",
         is_configurable=False,
-        doc_default="``~/.xonsh_history``",
+        doc_default="None",
+        type_str="path",
     )
     XONSH_HISTORY_MATCH_ANYWHERE = Var.with_default(
         False,
diff --git a/xonsh/shell.py b/xonsh/shell.py
--- a/xonsh/shell.py
+++ b/xonsh/shell.py
@@ -211,6 +211,7 @@ def __init__(self, execer, ctx=None, shell_type=None, **kwargs):
         builtins.__xonsh__.history = hist = xhm.construct_history(
             env=env.detype(), ts=[time.time(), None], locked=True
         )
+        env["XONSH_HISTORY_FILE"] = hist.filename
 
         shell_type = self.choose_shell_type(shell_type, env)
 


Test patch:
Repository: xonsh/xonsh

Issue:
Absolute paths to executables breaks bash completions
I came across this because I have a few aliases to make sure I'm running a given executable no matter what environment I'm in, e.g.

`aliases["git"] = "/usr/local/bin/git"`

## xonfig

<details>

```
$ xonfig
+------------------+---------------------+
| xonsh            | 0.9.26.dev7         |
| Git SHA          | c4e78435            |
| Commit Date      | Feb 9 10:54:17 2021 |
| Python           | 3.8.5               |
| PLY              | 3.11                |
| have readline    | True                |
| prompt toolkit   | 3.0.8               |
| shell type       | prompt_toolkit      |
| pygments         | 2.7.3               |
| on posix         | True                |
| on linux         | False               |
| on darwin        | True                |
| on windows       | False               |
| on cygwin        | False               |
| on msys2         | False               |
| is superuser     | False               |
| default encoding | utf-8               |
| xonsh encoding   | utf-8               |
| encoding errors  | surrogateescape     |
| on jupyter       | False               |
| jupyter kernel   | None                |
| xontrib          | []                  |
+------------------+---------------------+
```

</details>

## Expected Behavior
bash completions should work when we pass in an absolute path to an executable

```
$ /usr/local/bin/git st<TAB>
```

should show me git completions

## Current Behavior
```
$ /usr/local/bin/git st<TAB>
```

shows me path completions as a fallback

## What's going on
the bash completer function (and all the others, too) expect inputs in the form `(prefix, line, begidx, endidx, ctx)` -- only `prefix` and `line` are applicable here.

If I use the absolute path, then `line == "/usr/local/bin/git st"`, at which point the bash completer helper looks for an executable named `/usr/local/bin/git` which doesn't exist, i.e. it is parsing the path to the executable as the full executable.

We should do a `rstrip("/")[-1]` or similar to return the correct completions


## Steps to Reproduce
Try to complete a `git` command using the absolute path to your `git` executable

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**



Discussion:
Hey! waiting for https://github.com/xonsh/py-bash-completion/pull/17, and then we will use it in xonsh
Hey @daniel-shimon -- upstream merged and I also cut a new release with the changes.

Resolving patch:
diff --git a/xonsh/completers/bash_completion.py b/xonsh/completers/bash_completion.py
--- a/xonsh/completers/bash_completion.py
+++ b/xonsh/completers/bash_completion.py
@@ -15,7 +15,7 @@
 import subprocess
 import typing as tp
 
-__version__ = "0.2.6"
+__version__ = "0.2.7"
 
 
 @functools.lru_cache(1)
@@ -335,6 +335,7 @@ def bash_completions(
 
     splt = line.split()
     cmd = splt[0]
+    cmd = os.path.basename(cmd)
     idx = n = 0
     prev = ""
     for n, tok in enumerate(splt):


Test patch:
Repository: xonsh/xonsh

Issue:
xonfig wizard: list index out of range
Hi!

```python
XONSH_DEBUG=1 XONSH_SHOW_TRACEBACK=1 xonsh --no-rc
xonfig wizard
# Traceback (most recent call last):
#   File "/opt/miniconda/lib/python3.8/site-packages/xonsh/procs/proxies.py", line 506, in run
#     r = self.f(self.args, sp_stdin, sp_stdout, sp_stderr, spec, spec.stack)
#   File "/opt/miniconda/lib/python3.8/site-packages/xonsh/procs/proxies.py", line 298, in proxy_two
#     return f(args, stdin)
#   File "/opt/miniconda/lib/python3.8/site-packages/xonsh/aliases.py", line 703, in xonfig
#     return xonfig_main(args)
#   File "/opt/miniconda/lib/python3.8/site-packages/xonsh/xonfig.py", line 794, in xonfig_main
#     return XONFIG_MAIN_ACTIONS[ns.action](ns)
#   File "/opt/miniconda/lib/python3.8/site-packages/xonsh/xonfig.py", line 449, in _wizard
#     fname = env.get("XONSHRC")[-1] if ns.file is None else ns.file
#   File "/opt/miniconda/lib/python3.8/site-packages/xonsh/tools.py", line 206, in __getitem__
#     return _expandpath(self._l[item])
# IndexError: list index out of range
```

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**



Discussion:
Did you fix it? I can do a PR..
Nope, PR is welcome!

Resolving patch:
diff --git a/xonsh/xonfig.py b/xonsh/xonfig.py
--- a/xonsh/xonfig.py
+++ b/xonsh/xonfig.py
@@ -446,7 +446,8 @@ def make_xonfig_wizard(default_file=None, confirm=False, no_wizard_file=None):
 def _wizard(ns):
     env = builtins.__xonsh__.env
     shell = builtins.__xonsh__.shell.shell
-    fname = env.get("XONSHRC")[-1] if ns.file is None else ns.file
+    xonshrcs = env.get("XONSHRC", [])
+    fname = xonshrcs[-1] if xonshrcs and ns.file is None else ns.file
     no_wiz = os.path.join(env.get("XONSH_CONFIG_DIR"), "no-wizard")
     w = make_xonfig_wizard(
         default_file=fname, confirm=ns.confirm, no_wizard_file=no_wiz


Test patch:
Repository: xonsh/xonsh

Issue:
Unwanted "or/and" syntax highlighting in the beginning of the word
Hi!

![image](https://user-images.githubusercontent.com/1708680/111293377-42c02780-865a-11eb-90f7-3118781a95db.png)

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**



Discussion:
Hi @daniel-shimon! Please take a look. I think it's related to #3986
Haha seems like it!
Probably adding `\b` before and after each word in that regex will do the trick

Resolving patch:
diff --git a/xonsh/pyghooks.py b/xonsh/pyghooks.py
--- a/xonsh/pyghooks.py
+++ b/xonsh/pyghooks.py
@@ -1712,7 +1712,7 @@ def __init__(self, *args, **kwargs):
         ],
         "subproc": [
             include("mode_switch_brackets"),
-            (r"&&|\|\||and|or", Operator, "subproc_start"),
+            (r"&&|\|\||\band\b|\bor\b", Operator, "subproc_start"),
             (r'"(\\\\|\\[0-7]+|\\.|[^"\\])*"', String.Double),
             (r"'(\\\\|\\[0-7]+|\\.|[^'\\])*'", String.Single),
             (r"(?<=\w|\s)!", Keyword, "subproc_macro"),


Test patch:
Repository: xonsh/xonsh

Issue:
XSH singleton doesn't work with amalgamated (==production) xonsh
@jnoortheen 

```sh
# Ensure amalgamate.py is installed
which amalgamate.py
# Install xonsh in production mode (without '-e')
pip install '.[full]'
# Try running xonsh
xonsh
```

I'm getting:
```
AttributeError: 'XonshSession' object has no attribute 'completers'
Failed to load xontrib jedi.
Traceback (most recent call last):
  File "/home/daniel/testxonsh/venv/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 22082, in main
    args = premain(argv)
  File "/home/daniel/testxonsh/venv/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 22022, in premain
    env = start_services(shell_kwargs, args, pre_env=pre_env)
  File "/home/daniel/testxonsh/venv/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 21970, in start_services
    XSH.shell = Shell(execer=execer, **shell_kwargs)
  File "/home/daniel/testxonsh/venv/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 16365, in __init__
    XSH.history = hist = xhm.construct_history(
  File "/home/daniel/testxonsh/venv/lib/python3.8/site-packages/xonsh/history/__amalgam__.py", line 1241, in construct_history
    backend = env.get("XONSH_HISTORY_BACKEND")
AttributeError: 'NoneType' object has no attribute 'get'
Xonsh encountered an issue during launch
```



Discussion:
`XSH.load()` is not called. I am not sure why this is the case. Does the same run after removing all generated __amalgam__.py modules?
> `XSH.load()` is not called. I am not sure why this is the case. Does the same run after removing all generated **amalgam**.py modules?

No. Installing without amalgamation (`pip install -e`) works fine.
The issue seems to be that code from submodules imports and uses the non-amalgamated XSH, while amalgamated code uses the amalgamated XSH (I printed in a few places and indeed XSH in `xonsh/history/__amalgam__.py` and in `xonsh/__amalgam__.py` are two different objects).
Seems to me we need to disable amalgamating `built_ins.py` altogether

Resolving patch:
diff --git a/xonsh/__init__.py b/xonsh/__init__.py
--- a/xonsh/__init__.py
+++ b/xonsh/__init__.py
@@ -3,7 +3,7 @@
 
 # amalgamate exclude jupyter_kernel parser_table completion_parser_table parser_test_table pyghooks
 # amalgamate exclude winutils wizard pytest_plugin fs macutils pygments_cache
-# amalgamate exclude jupyter_shell proc
+# amalgamate exclude jupyter_shell proc built_ins
 import os as _os
 
 if _os.getenv("XONSH_DEBUG", ""):
@@ -86,8 +86,6 @@
         _sys.modules["xonsh.readline_shell"] = __amalgam__
         tracer = __amalgam__
         _sys.modules["xonsh.tracer"] = __amalgam__
-        built_ins = __amalgam__
-        _sys.modules["xonsh.built_ins"] = __amalgam__
         dumb_shell = __amalgam__
         _sys.modules["xonsh.dumb_shell"] = __amalgam__
         execer = __amalgam__


Test patch:
Repository: xonsh/xonsh

Issue:
!() doesn't capture information from subprocesses initiated in the background
## Steps to Reproduce

```
proc = !(ls &)
[1]+ running: ls & & (22822)
print(proc)
>>> None
```

## Expected behaviour
Something similar to this:
```
proc = !(ls)
proc
>>> CommandPipeline(
  stdin=<_io.BytesIO object at 0x7f333de561d0>,
  stdout=<_io.BytesIO object at 0x7f333de56180>,
  stderr=<_io.BytesIO object at 0x7f333de569f0>,
  pid=17977,
  returncode=0,
  args=['ls'],
  ...)
```

I can't see why a background process should return None. If I want to access said process' pid, then I have to do this:
```
import xonsh
xonsh.jobs.get_task(xonsh.jobs.tasks[-1])['pids']
```
Which is also a pretty slow expression (to write and to run).

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**



Discussion:

Resolving patch:
diff --git a/xonsh/procs/specs.py b/xonsh/procs/specs.py
--- a/xonsh/procs/specs.py
+++ b/xonsh/procs/specs.py
@@ -892,15 +892,14 @@ def run_subproc(cmds, captured=False, envs=None):
         # sure that the shell doesn't hang. This `pause_call_resume` invocation
         # does this
         pause_call_resume(proc, int)
-    # create command or return if backgrounding.
-    if background:
+    # now figure out what we should return
+    if captured == "object":
+        return command  # object can be returned even if backgrounding
+    elif background:
         return
-    # now figure out what we should return.
-    if captured == "stdout":
+    elif captured == "stdout":
         command.end()
         return command.output
-    elif captured == "object":
-        return command
     elif captured == "hiddenobject":
         command.end()
         return command


Test patch:
diff --git a/tests/procs/test_specs.py b/tests/procs/test_specs.py
--- a/tests/procs/test_specs.py
+++ b/tests/procs/test_specs.py
@@ -102,6 +102,23 @@ def test_capture_always(capfd, thread_subprocs, capture_always):
     assert exp in capfd.readouterr().out
 
 
+@skip_if_on_windows
+@pytest.mark.parametrize(
+    "captured, exp_is_none",
+    [
+        ("object", False),
+        ("stdout", True),
+        ("hiddenobject", True),
+        (False, True),
+    ],
+)
+def test_run_subproc_background(captured, exp_is_none):
+
+    cmds = (["echo", "hello"], "&")
+    return_val = run_subproc(cmds, captured)
+    assert (return_val is None) == exp_is_none
+
+
 @pytest.mark.parametrize("thread_subprocs", [False, True])
 def test_callable_alias_cls(thread_subprocs, xession):
     class Cls: