Repository: neovim
Issue: [race,linux] LSP textDocument/didChange is sent with invalid Unicode string, producing invalid JSON as result
Labels: bug,lsp

### Problem

When inserting Unicode from Xorg CLIPBOARD (or PRIMARY, not sure, I've configured them to sync) using `<S-Insert>` keypress (handled by X terminal app, I think) Neovim may (it's a race) send invalid `textDocument/didChange` JSON-RPC 2 request to LSP server.

The request is invalid because JSON string with Neovim's buffer contents ends with incomplete Unicode symbol. E.g. it may contain only first byte of multi-byte symbol before `"` which is supposed to close JSON string but instead that `"` may be handled as second byte of that broken Unicode symbol by LSP server's JSON parser.

I suppose this issue may depends on X terminal (I fail to reproduce it using NORMAL mode `p` to paste Unicode from X clipboard, it happens only when I press `<S-Insert>`). I was able to reproduce it in urxvt and xterm, but fail to reproduce in wezterm and kitty. I'm using xorg-server-21.1.13.

### Steps to reproduce

This repro uses LSP markdown-oxide just because it return error when receive invalid JSON-RPC 2.0 request, and this error is shown by Neovim as an indication of this issue. Some other LSP just ignore such an invalid request, which makes it harder to detect this issue (it still can be noticed in Neovim's `lsp.log`).

```
$ nvim --clean -u minimal.lua
:MasonInstall markdown-oxide
:qa
$ # Copy some amount of multi-byte Unicode symbols into X clipboard:
$ seq 40 | xargs -n 1 echo 󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤󰃤 | xsel -i
$ # File test.md does not exists, it's used only to attach LSP server
$ nvim --clean -u minimal.lua test.md
<Insert><S-Insert><S-Insert><S-Insert>…
```

On my system it's usually enough to press `<S-Insert>` a couple of times to trigger this issue and get this as an indication of successfully triggered issue:

```
LSP[markdown_oxide]: Error INVALID_SERVER_MESSAGE: {
  error = {                                                                                         
    code = -32700,                                                                                  
    message = "Parse error"                                                                         
  },                                                                                                
  jsonrpc = "2.0"                                                                                   
}                                                                                                   
```

Next you should check `~/.local/state/nvim/lsp.log` and last `textDocument/didChange` params will looks like this (last Unicode in a string is broken):

```
[DEBUG][2024-07-29 04:13:54] .../vim/lsp/rpc.lua:286	"rpc.send"	{
  jsonrpc = "2.0",
  method = "textDocument/didChange",
  params = {
    contentChanges = { {
        text = "ó°¤ó°¤ó°¤.........ó°¤ó°¤ 16\nó°\n"
      } },
    textDocument = {
      uri = "file:///home/powerman/.config/nvim/test.md",
      version = 11
    }
  }
}
```
[lsp.log](https://github.com/user-attachments/files/16406085/lsp.log)


### Expected behavior

Neovim should send only valid JSON to LSP servers, without incomplete Unicode symbols.

### Neovim version (nvim -v)

v0.10.0, v0.11.0-dev-512+gfe5030c05-dirty

### Vim (not Nvim) behaves the same?

N/A

### Operating system/version

Gentoo Linux

### Terminal name/version

rxvt-unicode-9.31, xterm-393

### $TERM environment variable

rxvt-unicode-256color, xterm

### Installation

Gentoo portage