Repository: xonsh/xonsh

Issue:
Windows: Don't imply PATH=.
In 48dac6e9cdaec8f20d7054aa44213fed6803a462, @byk added a few lines to what is now [found](https://github.com/xonsh/xonsh/blob/6245e460a6841b1bc51382265e4bb9afa8c9ec58/xonsh/commands_cache.py#L281-L286) in the command_cache module:

https://github.com/xonsh/xonsh/blob/6f8f82646e95ab5e626de7cf9b0dbf9717e2b2dd/xonsh/environ.py#L770-L772

It's not even obvious from that commit why the behavior was even relevant to enabling sudo.

In my opinion, `xonsh` shouldn't be adopting platform-specific behaviors from other shells but should instead provide a unified "xonsh" experience across platforms. Additionally, the presence of this platform-specific behavior complicates the implementation and adds cognitive load to maintenance.

And while it's It's true that `cmd.exe` will allow execution of bare executables in the current working directory, that's just a cmd.exe behavior. Powershell, for example, does not have that behavior:

```
PS C:\Users\jaraco> Get-ChildItem -File -Filter "*.exe" | Select-Object -ExpandProperty Name
rustup-init.exe
PS C:\Users\jaraco> rustup-init
rustup-init : The term 'rustup-init' is not recognized as the name of a cmdlet, function, script file, or operable program. 
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ rustup-init
+ ~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (rustup-init:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException


Suggestion [3,General]: The command rustup-init was not found, but does exist in the current location. Windows PowerShell does not load commands from the current location by default. If you trust this command, instead type: ".\rustup-init". See "get-help about_Command_Precedence" for more details.
```

xonsh should drop this legacy divergent behavior and replace it with a unified expectation that matches that of xonsh on other platforms and (incidentally) Powershell.

Discussion:
github-actions[bot]: **Warning!** No news item is found for this PR.
If this is an user facing change/feature/fix, please add a news item by copying the format from `news/TEMPLATE.rst`.

jaraco: Confirmed that 11d2e46f fixes #5469:

```
jaraco@bulletproof ~\draft @# ./rustup-init --version
rustup-init 1.27.1 (54dd3d00f 2024-04-24)
```

Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 78350f74..ea8df094 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -8,7 +8,7 @@ default_language_version:
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: 'v0.4.5'
+    rev: 'v0.4.7'
     hooks:
       - id: ruff
         args: [., --fix, --exit-non-zero-on-fix]
diff --git a/README.rst b/README.rst
index 0c37d331..74b3fb6a 100644
--- a/README.rst
+++ b/README.rst
@@ -65,6 +65,7 @@ Projects that use xonsh or compatible
 - `Regro autotick bot <https://github.com/regro/cf-scripts>`_: Regro Conda-Forge autoticker.
 - `zoxide <https://github.com/ajeetdsouza/zoxide>`_: A smarter cd command.
 - `any-nix-shell <https://github.com/haslersn/any-nix-shell>`_: xonsh support for the ``nix run`` and ``nix-shell`` environments of the Nix package manager.
+- `Starship <https://starship.rs/>`_: Cross-shell prompt.
 
 Jupyter-based interactive notebooks via `xontrib-jupyter <https://github.com/xonsh/xontrib-jupyter>`_:
 
diff --git a/docs/platform-issues.rst b/docs/platform-issues.rst
index fd253f44..d2fe7ccf 100644
--- a/docs/platform-issues.rst
+++ b/docs/platform-issues.rst
@@ -241,3 +241,28 @@ You can add aliases to your ``~/.xonshrc`` to have it always
 available when xonsh starts.
 
 
+Working Directory on PATH
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Windows users, particularly those coming from the ``cmd.exe`` shell,
+might be accustomed to being able to run executables from the current
+directory by simply typing the program name.
+
+Since version 0.16, ``xonsh`` follows the more secure and modern
+approach of not including the current working directory in the search
+path, similar to Powershell and popular Unix shells. To invoke commands
+in the current directory on any platform, include the current directory
+explicitly:
+
+.. code-block:: xonshcon
+
+    >>> ./my-program
+
+Although not recommended, to restore the behavior found in the
+``cmd.exe`` shell, simply append ``.`` to the ``PATH``:
+
+.. code-block:: xonshcon
+
+    >>> $PATH.append('.')
+
+Add that to ``~/.xonshrc`` to enable that as the default behavior.
diff --git a/docs/tutorial.rst b/docs/tutorial.rst
index c049ce50..17f4829a 100644
--- a/docs/tutorial.rst
+++ b/docs/tutorial.rst
@@ -1484,6 +1484,30 @@ Removing an alias is as easy as deleting the key from the alias dictionary:
    Otherwise, they may shadow the alias itself, as Python variables take
    precedence over aliases when xonsh executes commands.
 
+Callable alias and capturing
+----------------------------
+
+Callable aliases tend to be capturable. Only the explicitly denoted uncaptured subprocess
+operator ``$[]`` is uncapturable, and the subprocess's stdout passes directly
+through xonsh to the screen.
+
+.. code-block:: xonshcon
+
+    >>> @aliases.register('hunter')
+    ... def _hunter():
+    ...     print('catch me')
+    ...     echo if  # The same as `![echo if]`
+    ...     $[echo you]
+    ...     ![echo can]
+    >>> hunter
+    catch me
+    if
+    you
+    can
+
+    >>> $(hunter)
+    you
+    'catch me\nif\ncan\n'
 
 Anonymous Aliases
 -----------------
diff --git a/news/5477.rst b/news/5477.rst
new file mode 100644
index 00000000..86e1fc72
--- /dev/null
+++ b/news/5477.rst
@@ -0,0 +1,23 @@
+**Added:**
+
+* <news item>
+
+**Changed:**
+
+* Minor cleanup of ``commands_cache``, unifying behavior across platforms.
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* No longer is ``.`` implied for running commands on Windows. Instead the behavior is the same across platforms. Windows users will need to prefix ``./`` or ``.\`` to run commands from the current directory (#5476).
+
+**Fixed:**
+
+* Commands on Windows now honor the case as they appear on the file system (#5469).
+
+**Security:**
+
+* <news item>
diff --git a/news/fix_redir.rst b/news/fix_redir.rst
new file mode 100644
index 00000000..17633337
--- /dev/null
+++ b/news/fix_redir.rst
@@ -0,0 +1,23 @@
+**Added:**
+
+* <news item>
+
+**Changed:**
+
+* <news item>
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* <news item>
+
+**Fixed:**
+
+* Fixed redirect with python substitution e.g. ``echo 1 > @('/tmp/file')`` is working now.
+
+**Security:**
+
+* <news item>
diff --git a/news/tutorial_callias_capturing.rst b/news/tutorial_callias_capturing.rst
new file mode 100644
index 00000000..160977e2
--- /dev/null
+++ b/news/tutorial_callias_capturing.rst
@@ -0,0 +1,23 @@
+**Added:**
+
+* Added "Callable alias and capturing" to the tutorial.
+
+**Changed:**
+
+* <news item>
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* <news item>
+
+**Fixed:**
+
+* <news item>
+
+**Security:**
+
+* <news item>
diff --git a/pyproject.toml b/pyproject.toml
index 20cadeaf..6e0e1397 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -15,6 +15,9 @@ authors = [{ name = "Anthony Scopatz" }, { email = "scopatz@gmail.com" }]
 maintainers = [{ name = "Anthony Scopatz" }, { email = "scopatz@gmail.com" }]
 license = { text = "BSD 2-Clause License" }
 requires-python = ">=3.9"
+dependencies = [
+    "case-insensitive-dictionary; platform_system=='Windows'",
+]
 
 [tool.setuptools.dynamic]
 version = {attr = "xonsh.__version__"}
diff --git a/xonsh/commands_cache.py b/xonsh/commands_cache.py
index b86a85c1..07cc046c 100644
--- a/xonsh/commands_cache.py
+++ b/xonsh/commands_cache.py
@@ -18,6 +18,11 @@ from xonsh.lazyasd import lazyobject
 from xonsh.platform import ON_POSIX, ON_WINDOWS, pathbasename
 from xonsh.tools import executables_in
 
+if ON_WINDOWS:
+    from case_insensitive_dict import CaseInsensitiveDict as CacheDict
+else:
+    CacheDict = dict
+
 
 class _Commands(tp.NamedTuple):
     mtime: float
@@ -80,12 +85,7 @@ class CommandsCache(cabc.Mapping):
 
     def iter_commands(self):
         """Wrapper for handling windows path behaviour"""
-        for cmd, (path, is_alias) in self.all_commands.items():
-            if ON_WINDOWS and path is not None:
-                # All command keys are stored in uppercase on Windows.
-                # This ensures the original command name is returned.
-                cmd = pathbasename(path)
-            yield cmd, (path, is_alias)
+        return self.all_commands.items()
 
     def __len__(self):
         return len(self.all_commands)
@@ -99,15 +99,16 @@ class CommandsCache(cabc.Mapping):
         return len(self._cmds_cache) == 0
 
     def get_possible_names(self, name):
-        """Generates the possible `PATHEXT` extension variants of a given executable
-        name on Windows as a list, conserving the ordering in `PATHEXT`.
-        Returns a list as `name` being the only item in it on other platforms."""
-        if ON_WINDOWS:
-            pathext = [""] + self.env.get("PATHEXT", [])
-            name = name.upper()
-            return [name + ext for ext in pathext]
-        else:
-            return [name]
+        """Expand name to all possible variants based on `PATHEXT`.
+
+        PATHEXT is a Windows convention containing extensions to be
+        considered when searching for an executable file.
+
+        Conserves order of any extensions found and gives precedence
+        to the bare name.
+        """
+        extensions = [""] + self.env.get("PATHEXT", [])
+        return [name + ext for ext in extensions]
 
     @staticmethod
     def remove_dups(paths):
@@ -159,11 +160,10 @@ class CommandsCache(cabc.Mapping):
         # entries at the back.
         paths = tuple(reversed(tuple(self.remove_dups(env.get("PATH") or []))))
         if any(self._check_changes(paths)):
-            all_cmds = {}
+            all_cmds = CacheDict()
             for cmd, path in self._iter_binaries(paths):
-                key = cmd.upper() if ON_WINDOWS else cmd
                 # None     -> not in aliases
-                all_cmds[key] = (path, None)
+                all_cmds[cmd] = (path, None)
 
             # aliases override cmds
             for cmd in self.aliases:
@@ -178,9 +178,8 @@ class CommandsCache(cabc.Mapping):
                     # (path, False) -> has same named alias
                     all_cmds[override_key] = (all_cmds[override_key][0], False)
                 else:
-                    key = cmd.upper() if ON_WINDOWS else cmd
                     # True -> pure alias
-                    all_cmds[key] = (cmd, True)
+                    all_cmds[cmd] = (cmd, True)
             self._cmds_cache = all_cmds
         return self._cmds_cache
 
@@ -218,13 +217,9 @@ class CommandsCache(cabc.Mapping):
 
     def cached_name(self, name):
         """Returns the name that would appear in the cache, if it exists."""
-        if name is None:
-            return None
         cached = pathbasename(name) if os.pathsep in name else name
-        if ON_WINDOWS:
-            keys = self.get_possible_names(cached)
-            cached = next((k for k in keys if k in self._cmds_cache), None)
-        return cached
+        keys = self.get_possible_names(cached)
+        return next((k for k in keys if k in self._cmds_cache), name)
 
     def lazyin(self, key):
         """Checks if the value is in the current cache without the potential to
@@ -262,7 +257,6 @@ class CommandsCache(cabc.Mapping):
             Force return of binary path even if alias of ``name`` exists
             (default ``False``)
         """
-        # make sure the cache is up to date by accessing the property
         self.update_cache()
         return self.lazy_locate_binary(name, ignore_alias)
 
@@ -278,12 +272,6 @@ class CommandsCache(cabc.Mapping):
             (default ``False``)
         """
         possibilities = self.get_possible_names(name)
-        if ON_WINDOWS:
-            # Windows users expect to be able to execute files in the same
-            # directory without `./`
-            local_bin = next((fn for fn in possibilities if os.path.isfile(fn)), None)
-            if local_bin:
-                return os.path.abspath(local_bin)
         cached = next((cmd for cmd in possibilities if cmd in self._cmds_cache), None)
         if cached:
             (path, alias) = self._cmds_cache[cached]
@@ -329,18 +317,6 @@ class CommandsCache(cabc.Mapping):
         """
         name = self.cached_name(cmd0)
         predictors = self.threadable_predictors
-        if ON_WINDOWS:
-            # On all names (keys) are stored in upper case so instead
-            # we get the original cmd or alias name
-            path, _ = self.lazyget(name, (None, None))
-            if path is None:
-                return predict_true
-            else:
-                name = pathbasename(path)
-            if name not in predictors:
-                pre, ext = os.path.splitext(name)
-                if pre in predictors:
-                    predictors[name] = predictors[pre]
         if name not in predictors:
             predictors[name] = self.default_predictor(name, cmd0)
         predictor = predictors[name]
diff --git a/xonsh/procs/pipelines.py b/xonsh/procs/pipelines.py
index d210f826..4aba18a4 100644
--- a/xonsh/procs/pipelines.py
+++ b/xonsh/procs/pipelines.py
@@ -155,6 +155,7 @@ class CommandPipeline:
         self._stderr_prefix = self._stderr_postfix = None
         self.term_pgid = None
         self.suspended = None
+        self.output_format = self.spec.output_format
 
         background = self.spec.background
         pipeline_group = None
@@ -661,21 +662,21 @@ class CommandPipeline:
 
     def get_formatted_lines(self, lines):
         """Format output lines."""
-        format = XSH.env.get("XONSH_SUBPROC_OUTPUT_FORMAT", "stream_lines")
-        if format == "stream_lines":
+        fmt = self.output_format
+        if fmt == "stream_lines":
             if len(lines) == 1:
                 return lines[0].rstrip("\n")
             else:
                 return "".join(lines)
-        elif format == "list_lines":
+        elif fmt == "list_lines":
             if not lines:
                 return lines
             elif len(lines) == 1:
-                return lines[0].rstrip("\n")
+                return [lines[0].rstrip("\n")]
             else:
                 return [line.rstrip("\n") for line in lines]
-        elif callable(format):
-            return format(lines)
+        elif callable(fmt):
+            return fmt(lines)
 
     @property
     def output(self):
diff --git a/xonsh/procs/specs.py b/xonsh/procs/specs.py
index 267a6c6e..ee4e5d84 100644
--- a/xonsh/procs/specs.py
+++ b/xonsh/procs/specs.py
@@ -223,6 +223,9 @@ def _parse_redirects(r, loc=None):
 
 def _redirect_streams(r, loc=None):
     """Returns stdin, stdout, stderr tuple of redirections."""
+    if isinstance(loc, list):
+        raise Exception(f"Unsupported redirect: {r!r} {loc!r}")
+
     stdin = stdout = stderr = None
     no_ampersand = r.replace("&", "")
     # special case of redirecting stderr to stdout
@@ -417,6 +420,7 @@ class SubprocSpec:
         self.captured_stderr = None
         self.stack = None
         self.spec_modifiers = []  # List of SpecModifierAlias objects that applied to spec.
+        self.output_format = XSH.env.get("XONSH_SUBPROC_OUTPUT_FORMAT", "stream_lines")
 
     def __str__(self):
         s = self.__class__.__name__ + "(" + str(self.cmd) + ", "
@@ -671,7 +675,21 @@ class SubprocSpec:
         """Weave a list of arguments into a command."""
         resolved_cmd = []
         for c in self.cmd:
-            resolved_cmd += c if isinstance(c, list) else [c]
+            if (
+                isinstance(c, tuple)
+                and len(c) == 2
+                and isinstance(c[1], list)
+                and len(c[1]) == 1
+            ):
+                # Redirect case e.g. `> file`
+                resolved_cmd.append(
+                    (
+                        c[0],
+                        c[1][0],
+                    )
+                )
+            else:
+                resolved_cmd += c if isinstance(c, list) else [c]
         self.cmd = resolved_cmd
 
     def resolve_redirects(self):


Test patch:
diff --git a/tests/procs/test_specs.py b/tests/procs/test_specs.py
index edb9e912..e7832054 100644
--- a/tests/procs/test_specs.py
+++ b/tests/procs/test_specs.py
@@ -11,6 +11,7 @@ from xonsh.procs.posix import PopenThread
 from xonsh.procs.proxies import STDOUT_DISPATCHER, ProcProxy, ProcProxyThread
 from xonsh.procs.specs import (
     SpecAttrModifierAlias,
+    SpecModifierAlias,
     SubprocSpec,
     _run_command_pipeline,
     cmds_to_specs,
@@ -152,6 +153,7 @@ def test_capture_always(
 @skip_if_on_windows
 @pytest.mark.parametrize("captured", ["stdout", "object"])
 @pytest.mark.parametrize("interactive", [True, False])
+@pytest.mark.flaky(reruns=3, reruns_delay=2)
 def test_interrupted_process_returncode(xonsh_session, captured, interactive):
     xonsh_session.env["XONSH_INTERACTIVE"] = interactive
     xonsh_session.env["RAISE_SUBPROC_ERROR"] = False
@@ -186,8 +188,8 @@ def test_specs_with_suspended_captured_process_pipeline(
 @pytest.mark.parametrize(
     "cmds, exp_stream_lines, exp_list_lines",
     [
-        ([["echo", "-n", "1"]], "1", "1"),
-        ([["echo", "-n", "1\n"]], "1", "1"),
+        ([["echo", "-n", "1"]], "1", ["1"]),
+        ([["echo", "-n", "1\n"]], "1", ["1"]),
         ([["echo", "-n", "1\n2\n3\n"]], "1\n2\n3\n", ["1", "2", "3"]),
         ([["echo", "-n", "1\r\n2\r3\r\n"]], "1\n2\n3\n", ["1", "2", "3"]),
         ([["echo", "-n", "1\n2\n3"]], "1\n2\n3", ["1", "2", "3"]),
@@ -270,6 +272,20 @@ def test_spec_modifier_alias_tree(xession):
     assert spec.force_threadable is False
 
 
+@skip_if_on_windows
+def test_spec_modifier_alias_output_format(xession):
+    class SpecModifierOutputLinesAlias(SpecModifierAlias):
+        def on_modifer_added(self, spec):
+            spec.output_format = "list_lines"
+
+    xession.aliases["xlines"] = SpecModifierOutputLinesAlias()
+
+    cmds = [["xlines", "echo", "1\n2\n3"]]
+    specs = cmds_to_specs(cmds, captured="stdout")
+    (p := _run_command_pipeline(specs, cmds)).end()
+    assert p.output == ["1", "2", "3"]
+
+
 @pytest.mark.parametrize("thread_subprocs", [False, True])
 def test_callable_alias_cls(thread_subprocs, xession):
     class Cls:
@@ -353,6 +369,14 @@ def test_on_command_not_found_doesnt_fire_in_non_interactive_mode(xession):
     assert not fired
 
 
+def test_redirect_to_substitution(xession):
+    s = SubprocSpec.build(
+        # `echo hello > @('file')`
+        ["echo", "hello", (">", ["file"])]
+    )
+    assert s.stdout.name == "file"
+
+
 def test_partial_args_from_classmethod(xession):
     class Class:
         @classmethod
diff --git a/tests/test_commands_cache.py b/tests/test_commands_cache.py
index 61428922..a586b9b0 100644
--- a/tests/test_commands_cache.py
+++ b/tests/test_commands_cache.py
@@ -157,7 +157,6 @@ def test_commands_cache_predictor_default(args, xession, tmp_path):
 
 
 class Test_is_only_functional_alias:
-    @skip_if_on_windows
     def test_cd(self, xession):
         xession.aliases["cd"] = lambda args: os.chdir(args[0])
         xession.env["PATH"] = []
@@ -204,7 +203,31 @@ def test_update_cache(xession, tmp_path):
     assert file2.samefile(cached[basename][0])
 
 
-@skip_if_on_windows
+@pytest.fixture
+def faux_binary(tmp_path):
+    """
+    A fake binary in the temp path.
+
+    Uses mixed case so tests may make assertions about it.
+    """
+    binary = tmp_path / "RunMe.exe"
+    binary.touch()
+    binary.chmod(0o755)
+    return binary
+
+
+def test_find_binary_retains_case(faux_binary):
+    cache = CommandsCache({"PATH": []})
+    loc = cache.locate_binary(str(faux_binary))
+    assert faux_binary.name in loc
+
+
+def test_exes_in_cwd_are_not_matched(faux_binary, monkeypatch):
+    monkeypatch.chdir(faux_binary.parent)
+    cache = CommandsCache({"PATH": []})
+    assert cache.locate_binary(faux_binary.name) is None
+
+
 def test_nixos_coreutils(tmp_path):
     """On NixOS the core tools are the symlinks to one universal ``coreutils`` binary file."""
     path = tmp_path / "core"
diff --git a/tests/test_integrations.py b/tests/test_integrations.py
index b9a772a8..38fb0835 100644
--- a/tests/test_integrations.py
+++ b/tests/test_integrations.py
@@ -156,6 +156,34 @@ f e>o
         "The Truth is Out There\n",
         0,
     ),
+    # test redirecting to a python substitution
+    (
+        """
+def _f():
+    print('Wow Mom!')
+aliases['f'] = _f
+f > @('tttt')
+with open('tttt') as tttt:
+    s = tttt.read().strip()
+print('REDIRECTED OUTPUT: ' + s)
+""",
+        "REDIRECTED OUTPUT: Wow Mom!\n",
+        0,
+    ),
+    # test redirecting to a python substitution with p-string
+    (
+        """
+def _f():
+    print('Wow Mom!')
+aliases['f'] = _f
+f > @(p'tttt')
+with open('tttt') as tttt:
+    s = tttt.read().strip()
+print('REDIRECTED OUTPUT: ' + s)
+""",
+        "REDIRECTED OUTPUT: Wow Mom!\n",
+        0,
+    ),
     # test system exit in function alias
     (
         """
diff --git a/tests/test_pipelines.py b/tests/test_pipelines.py
index 1cf35d96..24f646f0 100644
--- a/tests/test_pipelines.py
+++ b/tests/test_pipelines.py
@@ -74,7 +74,6 @@ def patched_events(monkeypatch, xonsh_events, xonsh_session):
         ("!(echo hi | grep x)", "", "", ""),
     ),
 )
-@pytest.mark.flaky(reruns=3, reruns_delay=2)
 def test_command_pipeline_capture(cmdline, stdout, stderr, raw_stdout, xonsh_execer):
     pipeline: CommandPipeline = xonsh_execer.eval(cmdline)
     assert pipeline.out == stdout