Repository: neovim
Issue: Python host crash with vimspector
Labels: status:blocked-external,provider,bug-crash

### Neovim Version

(nvim) ben@BeniMac2020 neovim % ./build/bin/nvim --version NVIM v0.6.0-dev+158-gd088066fa Build type: Debug LuaJIT 2.1.0-beta3 Compilation: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -DNVIM_TS_HAS_SET_MATCH_LIMIT -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wmissing-prototypes -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fno-common -fdiagnostics-color=always -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -DMIN_LOG_LEVEL=1 -I/Users/ben/Development/neovim/build/config -I/Users/ben/Development/neovim/src -I/Users/ben/Development/neovim/.deps/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include -I/usr/local/opt/gettext/include -I/Users/ben/Development/neovim/build/src/nvim/auto -I/Users/ben/Development/neovim/build/include Compiled by ben@BeniMac2020.local  Features: +acl +iconv +tui See ":help feature-compile"     system vimrc file: "$VIM/sysinit.vim"   fall-back for $VIM: "/usr/local/share/nvim"  Run :checkhealth for more info

### Operating system/version:

macOS Big Sur (python 3.9.6 from pyenv built with PYTHON_CONFIGURE_OPTS="--enable-framework --with-address-sanitizer"

### Terminal name/version:

iTerm 2 3.4.8

### TERM environment variable

xterm-256color

### Installation

Local debug build is required for the repro to get asan error. Reproduces in latest master and 0.4.4 (from homebrew) at least.

### Steps to reproduce

I have two repros:

1. The issue [reported by the user](https://github.com/puremourning/vimspector/issues/441)
2. A minimal production of _an_ ASAN error which is triggered always when using the python provider

Unfortunately I have been unable to find a minimal reproduction that reliably crashes the python provider host for a stock python build, but I was able to trivial generate ASAN errors with a sanitised python build.

### Original issue

This is trivially reproduced as follows:

1. in a temporary directory `git clone https://github.com/puremourning/vimspector && cd vimspector`
2. `nvim -Nu support/minimal_vimrc`
3. `:VimspectorInstall vscode-bash-debug` (note this will only modify the current directory, it won't spam your system!)
4. `:edit support/test/bash/test_script`
5. `<F5>`
6. `1<CR><CR>` (select "Run Current Script"), (just accept the default for `args`)
7. `:VimspectorReset`

The result is as follows:

```


Error detected while processing function <SNR>42__OnCommandEvent[61]..provider#python3#Call:
line   18:
Error invoking 'python_execute' on channel 3 (python3-script-host):
ch 3 was closed by the client
Press ENTER or type command to continue
```

### Minimal reproduction of _a_ problem

Investigating this, I wrote up this gist: https://gist.github.com/puremourning/06991921b535d82228c9272ad3031db9

In order to reproduce _a_ possibly related problem:

1. Build Python 3.6.9 with `--with-address-sanitizer` (I used pyenv, as in `PYTHON_CONFIGURE_OPTS="--enable-framework --with-address-sanitizer pyenv install -g 3.6.9`. this creates a 3.6.9-debug pyenv.
2. make that your current python e.g. `pyenv shell 3.6.9-debug` (I actually used pyenv-virtualenv, but process is idem)
3. install pynvim `pip install neovim` (i used latest master with `pip install -e .`)
4. `nvim -Nu NONE -c 'py3 print("test")`

Result:

```

Error detected while processing function provider#python3#Call:
line   18:
Error invoking 'python_execute' on channel 3 (python3-script-host):
ch 3 was closed by the client
Press ENTER or type command to continue
```

To get the ASAN output, I had to apply this patch to the provider code. https://gist.github.com/puremourning/06991921b535d82228c9272ad3031db9#gistcomment-3866805





### Vim

yes, vim works fine

### Expected behavior

No python provider crash, vimspector and minimal example work.

### Actual behavior

As above.

the ASAN report for the minimal case is: https://gist.github.com/puremourning/d4f9ed79f39633b0e6837c1a0e7977e8

It's not possible to get the ASAN output for the vimspector reproduction because the provider crashes as soon as it is used, so it isn't possible to get far enough. It might be possible by adding CFLAGS='-fno-sanitize-recover=memory` when building python, though I have not done this.