Repository: vscode-neovim/vscode-neovim (TypeScript)

Issue:
Bug: Visual mode for files makes letters appear in the line that is selected
Have disabled all other plugins expect the colorscheme.

https://user-images.githubusercontent.com/7727602/215477372-ce102f74-c17e-4ac6-8b82-3ab926d48320.mov


bug: ghost cursor
### Did you check docs and existing issues?

- [X] I have read all the vscode-neovim docs
- [X] I have searched the existing issues of vscode-neovim
- [X] I can reproduce the issue with an empty init.vim/init.lua and no other vscode extensions (when applicable)

### Neovim version (nvim -v)

NVIM v0.9.1

### Operating system/version

Fedora 38

### Describe the bug

When switching between file editor and jupyter interactive windows multiple cursors will appear in the editor and remain in one spot until the session is reloaded. The issue only seems to happen if a visual selection is active in the editor when switching to the interactive window.

![image](https://github.com/vscode-neovim/vscode-neovim/assets/12448681/f4863a16-d8b3-4acc-aedf-8b73417f6658)

In the image above only the second cursor is the "real" one.

### Steps To Reproduce

1. Select a block of code using visual mode
2. Without exiting the visual mode switch to interactive window.
3. Return to your editor window

### Expected Behavior

No ghost cursor artifacts in the editor window.
bug: Weird virtual characters
### Did you check docs and existing issues?

- [X] I have read all the vscode-neovim docs
- [X] I have searched the existing issues of vscode-neovim
- [X] I can reproduce the issue with an empty init.vim/init.lua and no other vscode extensions (when applicable)

### Neovim version (nvim -v)

NVIM v0.9.1

### Operating system/version

Pop!_OS 22.04 LTS x86_64

### Describe the bug

when I enter visual mode and select some lines I find some weird characters at the end of some lines.

as you see in the following picture

![bug](https://github.com/vscode-neovim/vscode-neovim/assets/99615534/e8c04dbd-794a-486f-8fd1-dbcf3994b69c)

__NOTE__ I did not face this bug in version `0.3.2`

### Steps To Reproduce

1. go to visual mode
2. just select some lines 
3. sometimes these characters appear and sometimes not, so try step 1 and 2 until you find them

### Expected Behavior

What I expect to find that when I go to visual mode and select some lines, I do not find these weird characters


Discussion: I've been seeing this for a while too, possibly since the last update?

Perhaps worth noting is this seems to be cosmetic — the selection will look garbled, but as far as I can tell the backing buffer has the right selection, since e.g. `vi(y` still works as expected and yank the selection.
same issue, but looks like this
<img width="475" alt="image" src="https://user-images.githubusercontent.com/84488223/216487955-29852665-2ded-4651-b95f-b13fdccdbcae.png">

I have the same issue, but only with `.go` files.
@oleynikandrey it is actually true for me. I did not notice it but this is for `.go` files for me, updated the description
Hmm, the example @b4ljk posted doesn't look like Go, but I think it's true for me as well that most of the times I've seen it has been Go — I wonder if certain text decorations used by the Go extension make it more likely to occur? Offhand, the only ones I can think of:

- inlay type/parameter hints
- semantic token highlighting
- code lenses (e.g. `Run Test`, `Debug Test`)

Of those, none of them are unique to the Go extension but perhaps some combination of all of them causes it. It doesn't seem to be trivially reproducible in my case (reloading window seems to clear the messed up state). 

In most other languages, I use spaces for indentation, whereas Go uses tabs — perhaps that could be related too?
il keep it at all files again and see we can get to the bottom of this. I have no clue as I have to restart and reinstall all plugins but it appears as soon as I do it
I tried the nightly builds for copilot and vscode and now it seems to work. but it might appear again. like arnold says "I will be back"
I may be running into the same issue. On first insert the only suggestion is the letter 'i' (consistently):
<img width="1316" alt="Screenshot 2023-06-01 at 1 27 49 PM" src="https://github.com/vscode-neovim/vscode-neovim/assets/1072638/f570ab87-a61a-43b5-a3c1-22aa74339542">

Hitting <tab> inserts an actual tab the first time, but if I backspace back to the first of the line, it gets the suggestion nearly right, but with weird extra characters:
<img width="1316" alt="Screenshot 2023-06-01 at 1 27 57 PM" src="https://github.com/vscode-neovim/vscode-neovim/assets/1072638/9d38a943-1cf7-471d-9a82-6ba244a4e1a2">

And sometimes the chars are at the end of the suggestion (conspicuously the same char combination):
<img width="1316" alt="Screenshot 2023-06-01 at 1 28 18 PM" src="https://github.com/vscode-neovim/vscode-neovim/assets/1072638/b555bf5d-8679-415d-b397-623cb32b2d67">

Interestingly, if I start typing the suggestion, the weird trailing chars in one case stop being displayed:

<img width="899" alt="Screenshot 2023-06-01 at 1 47 43 PM" src="https://github.com/vscode-neovim/vscode-neovim/assets/1072638/289991c9-8180-4d54-b22c-34b13b71b0f1">
<img width="882" alt="Screenshot 2023-06-01 at 1 47 59 PM" src="https://github.com/vscode-neovim/vscode-neovim/assets/1072638/648eb411-d299-48e0-bbfa-eb056c13604f">

Note that for these screenshots, I was using my standard `.vimrc`, as evidenced by the nvim customizations in the vscode built-in terminal. I also tried with an empty `.vimrc` and no `~/.vim/` folder to make sure there was nothing in my n/vim settings that were introducing weird characters or anything. I also disabled every extension (all of 'em) except Copilot and Neovim.
Fwiw this is caused by bugs in the highlighting/extmark code. It is cosmetic only, but obviously annoying. 
I'm not sure if this is related, but I'm also seeing a similar issue when the vscode-neovim cursor is shown in a comment that has a wrapping line (Using `vscode-pull-request-github` in this case).

https://github.com/vscode-neovim/vscode-neovim/assets/11131775/dfcd385b-1f1a-47cc-9e27-282e5623647f

Is this the same, or should I file a separate issue for this cursor misalignment?
@ian-h-chamberlain honestly I feel like this is an issue with vscode, we don't touch typing or cursor in insert mode
> @ian-h-chamberlain honestly I feel like this is an issue with vscode, we don't touch typing or cursor in insert mode

@theol0403 looks like you're right — I can reproduce it without `vscode-neovim` enabled and I think it might actually have to do with some custom CSS I am using. Seems unrelated, sorry for the noise.
@ian-h-chamberlain @eleijonmarck  The problem is back again in 0.4.0 (was working fine in 0.3.2). It seems that the problem does not only exist in Go but in the other languages where the indent is set to tab as well
<img width="483" alt="image" src="https://github.com/vscode-neovim/vscode-neovim/assets/27771872/146e698c-fe76-4d72-b669-4e134101711c">
From the image above, I tried to change the indent on line no.3, from space to tab, inside a js file and select the whole function. It turned out that the letters only appear on line no.3.

I also tried to select the whole .go file, and only the lines without the indent(tab) have no letters appearing in the line.


I've narrowed down one source of the visual bug.

Apparently, when a custom hl group (visual fake cursor) is placed on top of a visual selection, the `listchars` disappear for that cell. Because of our listchars hack, the tab isn't being counted properly. Not sure if this is upstream or not yet.
Can confirm that I'm having the sampe problem. It seems that the issue arises only when the selection starts before tab characters.
If I enter the visual selection from the middle of the first line everything seems to be rendered well.

![tabs_behaviour](https://github.com/vscode-neovim/vscode-neovim/assets/29178712/5bb79fb1-e76e-4ce5-aa01-0fdbd965a1d0)


This might be a duplicate of #1352.
> This might be a duplicate of #1352.

The issue looks similar except I do not use colorcolumn anywhere in my settings. My init.vim is empty.
After some more testing I also see issues similar to #756. After switching to the jupyter interactive window the cursor is "stuck" in normal mode and then there will be a new cursor artifact when returning to the editor.
This is caused by the fake visual cursor not being properly hidden on buffer change. 

I'm on Ubuntu 22.04.3, nvim v0.9.1 and am getting the same issue. In my case, the extra virtual character is always a repeat of the last character from the line it appears on.

I have noticed that when I do visual line (V) I sometimes see a virtual copy of every character in the line, but shifted 1 space to the right. I don't know if this is an unrelated bug or not, but thought I should mention it.

This bug started around 2 - 3 weeks ago for me after an update to the vscode-neovim plugin.

![image](https://github.com/vscode-neovim/vscode-neovim/assets/21698432/c69c6df4-63b0-4905-8c45-10dd6529d114)

|                           |              |
| --------------------- | ---------- |
| Ubuntu              | 22.04.3  |
| nvim                  | v0.9.1    |
| VSCode            | 1.81.0    |
| vscode-neovim | 0.4.1      |


Hopefully this all proves helpful.

Looks like they recently released a major version [v0.4.0](https://github.com/vscode-neovim/vscode-neovim/releases/tag/v0.4.0) which introduce several breaking changes and features related to cursor.

I try rollback to 0.3.2 and everything works as usual. So i decide to use 0.3.2 until 0.4.x is stable.
getting this issue too on macOS.
<img width="338" alt="Screenshot 2023-08-10 alle 23 37 16" src="https://github.com/vscode-neovim/vscode-neovim/assets/45905658/369528b8-6a6b-48cf-8898-3df92c7209ec">

after downgrading to 0.3.2:
<img width="156" alt="Screenshot 2023-08-10 alle 23 38 22" src="https://github.com/vscode-neovim/vscode-neovim/assets/45905658/4ad3e1c5-8c29-4506-a79c-202afa2f2b6f">
looks a bit ugly but it works at least without ghost characters.

by the way, it doesn't do it in every language.
here are some examples (v0.3.2):
* JSON <img width="470" alt="Screenshot 2023-08-10 alle 23 43 09" src="https://github.com/vscode-neovim/vscode-neovim/assets/45905658/97394368-0d5b-4461-baec-6bf6c212fc43">
* Python <img width="147" alt="Screenshot 2023-08-10 alle 23 46 05" src="https://github.com/vscode-neovim/vscode-neovim/assets/45905658/5de7ccc9-dddd-4428-9c6f-174e7aa8a5a3">
* Typescript <img width="141" alt="Screenshot 2023-08-10 alle 23 47 06" src="https://github.com/vscode-neovim/vscode-neovim/assets/45905658/5bb9eda1-2284-4c3f-b8ce-62042a0074f8">

Seems like a go specific issue? very strange
I noticed something more: basically it struggles with Go files specifically with tab size 4 (gofmt default) which I noticed you are using too. for now I'll stick with 0.3.2 so it at least doesn't leave weird artifacts around the place (just a bit ugly)
It is really very strange. I tried do reproduce this behavior with rust and python but I can't :(:

1. rust

![image](https://github.com/vscode-neovim/vscode-neovim/assets/99615534/6e3b958d-a67f-4921-b89f-b724f98b7193)

2. python

![image](https://github.com/vscode-neovim/vscode-neovim/assets/99615534/cf096421-d3a5-472c-ab4d-29ca1dfd2e98)


@Tesohh 
I tried with rust and javascript, it worked fine. It only happens in go.

Especially after I yank something.

<img width="533" alt="CleanShot 2023-08-11 at 11 29 37@2x" src="https://github.com/vscode-neovim/vscode-neovim/assets/56581242/2f76d877-463d-4131-8a5f-d6e4a31f7d1e">



same issue on windows (still in go):
![image](https://github.com/vscode-neovim/vscode-neovim/assets/45905658/6ebd2f75-8a98-40c4-ab94-10baf89d8f7e)

I've been able to re-create with PHP, JS, and a txt file.
Was not able to re-create in Rust, Python, Perl, C, or JSON.
I don't have any Go code handy to test on.

The txt file has ANSI terminal escapes in it though (results from `grep --color=always` intended to be played back with `less -R`), so I don't know how much of the issue with this file is due to vscode/nvim trying to parse through those.

I have noticed that I struggle to re-create this in my smaller files ( x < 100 lines), though I'm not sure how much of this is luck.
I also found that restarting vscode can cause this issue to go away for a bit.
> getting this issue too on macOS.
> <img alt="Screenshot 2023-08-10 alle 23 37 16" width="338" src="https://user-images.githubusercontent.com/45905658/259879894-369528b8-6a6b-48cf-8898-3df92c7209ec.png">

Similar to @Tesohh's. On macOS, It looks like there are holes in the screen. And I noticed that these holes are moving with the screen scrolling.


https://github.com/vscode-neovim/vscode-neovim/assets/45259230/daf2efe1-64a9-43e8-bf54-6c6137d815f8



NVIM v0.9.1, VSCode Neovim v0.4.1
I've seen the same thing on Fedora 38, with neovim 0.9.1 and plugin 0.4.2. In my case, it occurs when I have a YAML file that has additional requirements beyond YAML (specifically: Helm template or ansible playbook). It seems to be an interaction with ALE, and specifically when ALE is installed via OS-level package.

In my case, I work around it by installing ALE in my user directory and skipping it the "normal" way (in init.vm via checking the vscode variable, and not enabling it).
Also facing this issue on Ubuntu 22.04 with latest VSCode version at the moment and latest VScode-Neovim extension version



Fix (patch):
diff --git a/runtime/lua/vscode-neovim/cursor.lua b/runtime/lua/vscode-neovim/cursor.lua
index dc549b897..e5d73ab22 100644
--- a/runtime/lua/vscode-neovim/cursor.lua
+++ b/runtime/lua/vscode-neovim/cursor.lua
@@ -90,6 +90,8 @@ function M.highlight_fake_cursor()
     local line = vim.fn.line(".")
     local col = vim.fn.col(".")
     local ch = util.get_char_at(line, col) or " "
+    -- !nvim won't send tab, so we use ❥ instead
+    ch = ch == '\t' and '❥' or ch
     M.fake_cursor = vim.api.nvim_buf_set_extmark(0, M.fake_ns, line - 1, col - 1,
       { virt_text = { { ch, "Cursor" } }, virt_text_pos = "overlay", hl_mode = "combine", priority = 65534 })
   end
diff --git a/src/buffer_manager.ts b/src/buffer_manager.ts
index 5c47af814..a3d48b667 100644
--- a/src/buffer_manager.ts
+++ b/src/buffer_manager.ts
@@ -533,10 +533,12 @@ export class BufferManager implements Disposable, NeovimRedrawProcessable, Neovi
             });
         }
 
-        const requests: [string, unknown[]][] = [
+        const tabOptions: [string, unknown[]][] = [
             ["nvim_buf_set_option", [bufId, "expandtab", insertSpaces]],
             ["nvim_buf_set_option", [bufId, "tabstop", tabSize]],
             ["nvim_buf_set_option", [bufId, "shiftwidth", tabSize]],
+        ];
+        const requests: [string, unknown[]][] = [
             // fill the buffer
             ["nvim_buf_set_lines", [bufId, 0, -1, false, lines]],
             // set vscode controlled flag so we can check it neovim
@@ -552,8 +554,14 @@ export class BufferManager implements Disposable, NeovimRedrawProcessable, Neovi
             ["nvim_buf_set_option", [bufId, "buftype", "nofile"]],
             // list buffer
             ["nvim_buf_set_option", [bufId, "buflisted", true]],
+            // nvim_buf_set_name will do filetype detection
+            // we must override tab options after vim initializes defaults
+            ...tabOptions,
         ];
         await callAtomic(this.client, requests, this.logger, LOG_PREFIX);
+        // Debugging through breakpoints reveals that in some cases the indentation options are overridden again.
+        // It is currently possible to work around this issue with a separate request
+        await callAtomic(this.client, tabOptions, this.logger, LOG_PREFIX);
         // Looks like need to be in separate request
         if (!this.isExternalTextDocument(document)) {
             await this.client.callFunction("VSCodeClearUndo", bufId);
diff --git a/src/highlight_provider.ts b/src/highlight_provider.ts
index 3483b1c2c..3d0eef769 100644
--- a/src/highlight_provider.ts
+++ b/src/highlight_provider.ts
@@ -186,6 +186,9 @@ export class HighlightProvider {
         let hasUpdates = false;
 
         for (const [ctext, hlId, repeat] of cells) {
+            if (hlId != null) {
+                cellHlId = hlId;
+            }
             let text = ctext;
 
             // 2+bytes chars (such as chinese characters) have "" as second cell
@@ -196,9 +199,6 @@ export class HighlightProvider {
             if (text === "♥") {
                 continue;
             }
-            if (hlId != null) {
-                cellHlId = hlId;
-            }
 
             const listCharsTab = "❥";