Repository: neovim
Issue: Fixing `'shell'` on Windows, properly
Labels: bug,platform:windows,terminal,has:plan,system

### Problem

See https://github.com/neovim/neovim/issues/16957 for some of the existing issues with using a Unix shell in Neovim on Windows. 

I've dug into it now, seeing as no workaround really managed to fully take, and here's what I was able to find.

The root of evil is the fact that `CreateProcessA` takes some long string as the command line. The Microsoft C runtime takes that string, runs it through splitting logic described [here](https://learn.microsoft.com/en-us/cpp/c-language/parsing-c-command-line-arguments?view=msvc-170), then passes the processed result to `main`. The result of that processing is what `'shell'` ultimately receives.

Vim on the other hand, wants to be able to think about running programs in terms of regular C `argv` arrays.

Assuming `shell=bash`, `shellcmdflag=-c` and `shellxquote="`, running `!ls -l`, produces the argument vector `[bash, -c, "ls -l"]` (note the quotes around `ls -l`). The resulting argument vector is then [passed to libuv](https://github.com/neovim/neovim/blob/master/src/nvim/event/libuv_process.c#L22), which then [reconstitutes the entire thing back to a single string](https://github.com/libuv/libuv/blob/v1.x/src/win/process.c#L521) so it can pass it to `CreateProcessA`.

This is where the issue arises. libuv expects an *unquoted* argument vector (`[bash, -c, ls -l]`, no quotes), as you would pass to `execve` on Unix, but gets `[bash, -c, "ls -l"]` instead. This causes it to *quote the quoted string* on Windows, which ultimately causes bash to receive the command `"ls -l"` (one shell word, with quotes) instead of `ls -l` (two shell words, no quotes).

So, basically, there are two places in the chain that are trying to make the argument vector "windows-friendly": Vim's `shell_build_argv`, and libuv's `make_program_args`. Only one of the two should be done. To only use vim's behaviour, it is probably sufficient to change [libuv_process_spawn](https://github.com/neovim/neovim/blob/master/src/nvim/event/libuv_process.c#L32) to always pass `UV_PROCESS_WINDOWS_VERBATIM_ARGUMENTS`. Since `cmd.exe` already has to use this behaviour since its argument parsing does *not* follow that of the C runtime (i.e. all other programs), this seems like it will cause the least amount of breakage.

To only follow libuv's behaviour (an arguably more sound strategy but that may not work with cmd.exe), we'd simply need to skip `shell_xescape_xquote` (or set both `shellxescape` and `shellxquote` to the empty string?).

Whichever the case may be, it's pretty clear that Neovim's current behaviour on Windows is incorrect. Double quoting like this shouldn't occur, and having two completely different implementations both doing argv-sanitation depending on context is bound to lead to issues. (As it did very recently for Rust)

There's also some other issues. For example, Neovim tries to pass `2>&1 | tee <tempfile>` to the shell. However, the Windows-style backslash-using tempfile path will cause the shell to misinterpret something like `C:\somedir\tmp\` to contain a `\t` (tab) (at least in zsh) and cause it to spit out the file in the wrong place, causing errors in `:make`, `:grep`, etc. Easiest solution would be to change the path to use `/` as the separator (possibly in accordance with `'shellslash'`, which doesn't apply to the temporary file arguments currently) (though this will also not work for UNC paths), or to use something like `cygpath` if available (most unix shells on Windows are cygwin/msys2 based).


### Steps to reproduce

nvim --clean
:set shell=C:/path/to/bash.exe shellcmdflag=-c
:!echo test <- fails
:set shellxquote=
:!echo test <- works
(and many other such cases)

### Expected behavior

We should figure out a way to fix this properly, once and for all.

### Neovim version (nvim -v)

master

### Vim (not Nvim) behaves the same?

Have not verified, but it seems to be Neovim specific

### Operating system/version

Windows 10/11

### Terminal name/version

wezterm, but doesn't matter (not TUI related)

### $TERM environment variable

xterm-256color

### Installation

GitHub nightly releases