Repository: microsoft/terminal (language: cpp)

Issue:
ghost chars when select CJK chars in vim visual mode
### Windows Terminal version

1.22.10731.0

### Windows build number

10.0.26100.0

### Other Software

vim 9.1
pwsh 7.5.0

### Steps to reproduce

create a file include CJK chars like below
```
中文内容
にほんご
```

use vim open it, and select one line in visual mode

### Expected Behavior

normal select line with fact chars

below is a normal result in conhost
![Image](https://github.com/user-attachments/assets/1ff0968f-8137-4b3c-b717-6ccf0dcda578)

### Actual Behavior

![Image](https://github.com/user-attachments/assets/ff29ef83-cb7b-4888-8184-7dc8aa2b89e0)

display some ghost chars after end of line


Discussion / hints:
this bug do not appear in wt version 1.21

and i have tried 1.23, this bug also appears
Thanks for the excellent repro! When I reproduce this, I get the following output from VIM during the selection (annotated by me):
```
␛[?25l              # Hide cursor
␛[0 q               # Reset cursor style
␛[1;1H              # Goto the selected line of text
␛[30;31H            # Goto status bar
␛7                  # Save cursor position
␛[30;31H            # Goto status bar
␛[0m                # Reset color
1-3                  # Print status bar
␛8                  # Restore cursor position
␛7                  # Save cursor position
␛[30;31H            # Goto status bar
␛[0m                # Reset color
2-6                  # Print status bar
␛8                  # Restore cursor position
␛[30;1H             # Goto status bar
␛[30;34H            # Goto status bar
␛[30;1H             # Goto status bar
␛[1;3H              # Goto selected line of text
␛[1;1H              # Goto selected line of text
␛[1;1H              # Goto selected line of text
␛[8m                # Make text invisible
␛7                  # Save cursor position
␛[1;1H              # Goto the selected line of text
␛[0;100m            # Set selection color
中中                 <-- ⚠️ VIM prints the char twice! ⚠️
␛8                  # Restore cursor position
␛7                  # Save cursor position
␛[1;1H              # Goto the selected line of text
␛[0;100m            # Set selection color
中                   <-- ⁉️ VIM prints the char again? ⁉️
␛8                  # Restore cursor position
␛[1;1H              # Goto the selected line of text
␛[1;3H              # Goto the selected line of text
␛[0m                # Reset color
␛[7m                # Invert colors
␛[0m                # Reset color
␛[1;1H              # Goto the selected line of text
␛[30;43H            # Goto status bar
␛7                  # Save cursor position
␛[30;43H            # Goto status bar
␛[0m                # Reset color
1                    # Print status bar
␛8                  # Restore cursor position
␛7                  # Save cursor position
␛[30;43H            # Goto status bar
␛[0m                # Reset color
4-3                  # Print status bar
␛8                  # Restore cursor position
␛[30;1H             # Goto status bar
␛[30;46H            # Goto status bar
␛[30;1H             # Goto status bar
␛[1;3H              # Goto the selected line of text
␛[?25h              # Show cursor
␛[0 q               # Reset cursor style
␛[<0;3;1m           # Reset colors and then make text italic and bold
␛[?25l              # Hide cursor
␛[0 q               # Reset cursor style
␛[?25h              # Show cursor
␛[0 q               # Reset cursor style
```

I do have to say, it's shocking to me how "chatty" vim is. I did not expect that at all.

More importantly though, it's vim itself which prints the wide glyph twice, so it's probably not a "bug" in Windows Terminal per se (strictly speaking), but rather an unintentional regression that we may have caused with one of the recent modernizations. For instance, #16916.

Edit: Probably a fallout from pass through mode. For… reasons? vim doesn't use VT for many operations on Windows and so it may be running into an edge case when trying to insert wide glyphs into narrow cells which we forbid now.
>  vim doesn't use VT for many operations on Windows and so it may be running into an edge case when trying to insert wide glyphs into narrow cells which we forbid now.

is there any setting to prevent this issue in current version?

> is there any setting to prevent this issue in current version?

Unfortunately not. You can only either use vim through MSYS2 or through WSL. Otherwise, you can of course downgrade to 1.21 for the time being.

Fix patch:
diff --git a/src/host/_output.cpp b/src/host/_output.cpp
index a075f5a303c..fda7ae2792b 100644
--- a/src/host/_output.cpp
+++ b/src/host/_output.cpp
@@ -110,15 +110,25 @@ static FillConsoleResult FillConsoleImpl(SCREEN_INFORMATION& screenInfo, FillCon
             switch (mode)
             {
             case FillConsoleMode::WriteAttribute:
+            {
                 for (; columns < columnsAvailable && inputPos < lengthToWrite; ++columns, ++inputPos)
                 {
-                    infoBuffer[columns].Attributes = input[inputPos];
+                    // Overwrite all attributes except for the lead/trail byte markers.
+                    // Those are used by WriteConsoleOutputWImplHelper to correctly serialize the input.
+                    constexpr auto LT = COMMON_LVB_LEADING_BYTE | COMMON_LVB_TRAILING_BYTE;
+                    auto& attributes = infoBuffer[columns].Attributes;
+                    attributes = (input[inputPos] & ~LT) | (attributes & LT);
                 }
                 break;
+            }
             case FillConsoleMode::FillAttribute:
                 for (const auto attr = input[0]; columns < columnsAvailable && inputPos < lengthToWrite; ++columns, ++inputPos)
                 {
-                    infoBuffer[columns].Attributes = attr;
+                    // Overwrite all attributes except for the lead/trail byte markers.
+                    // Those are used by WriteConsoleOutputWImplHelper to correctly serialize the input.
+                    constexpr auto LT = COMMON_LVB_LEADING_BYTE | COMMON_LVB_TRAILING_BYTE;
+                    auto& attributes = infoBuffer[columns].Attributes;
+                    attributes = (attr & ~LT) | (attributes & LT);
                 }
                 break;
             case FillConsoleMode::WriteCharacter:
diff --git a/src/host/ut_host/VtIoTests.cpp b/src/host/ut_host/VtIoTests.cpp
index 3ad87891a28..e557ed517d2 100644
--- a/src/host/ut_host/VtIoTests.cpp
+++ b/src/host/ut_host/VtIoTests.cpp
@@ -52,6 +52,16 @@ static constexpr std::wstring_view s_initialContentVT{
     // clang-format on
 };
 
+static constexpr std::wstring_view s_initialContentVTWide{
+    // clang-format off
+    L""
+    sgr_red("〇") sgr_blu("一") sgr_red("二") sgr_blu("三") "\r\n"
+    sgr_red("四") sgr_blu("五") sgr_red("六") sgr_blu("七") "\r\n"
+    sgr_blu("八") sgr_red("九") sgr_blu("十") sgr_red("百") "\r\n"
+    sgr_blu("千") sgr_red("万") sgr_blu("億") sgr_red("兆")
+    // clang-format on
+};
+
 class ::Microsoft::Console::VirtualTerminal::VtIoTests
 {
     BEGIN_TEST_CLASS(VtIoTests)
@@ -71,11 +81,11 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
         return { &rxBuf[0], read };
     }
 
-    void setupInitialContents() const
+    void setupInitialContents(bool wide) const
     {
         auto& sm = screenInfo->GetStateMachine();
         sm.ProcessString(L"\033c");
-        sm.ProcessString(s_initialContentVT);
+        sm.ProcessString(wide ? s_initialContentVTWide : s_initialContentVT);
         sm.ProcessString(L"\x1b[H" sgr_rst());
     }
 
@@ -277,7 +287,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
 
     TEST_METHOD(WriteConsoleOutputAttribute)
     {
-        setupInitialContents();
+        setupInitialContents(false);
 
         static constexpr std::array payload{ red, blu, red, blu };
         static constexpr til::point target{ 6, 1 };
@@ -295,7 +305,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
 
     TEST_METHOD(WriteConsoleOutputCharacterW)
     {
-        setupInitialContents();
+        setupInitialContents(false);
 
         size_t written = 0;
         std::string_view expected;
@@ -354,7 +364,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
         actual = readOutput();
         VERIFY_ARE_EQUAL(expected, actual);
 
-        setupInitialContents();
+        setupInitialContents(false);
 
         // Writing at the start of a line.
         THROW_IF_FAILED(routines.FillConsoleOutputAttributeImpl(*screenInfo, red, 3, { 0, 0 }, cellsModified, false));
@@ -388,6 +398,25 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
         VERIFY_ARE_EQUAL(expected, actual);
     }
 
+    TEST_METHOD(FillConsoleOutputAttributeWide)
+    {
+        setupInitialContents(true);
+
+        size_t cellsModified = 0;
+        std::string_view expected;
+        std::string_view actual;
+
+        // Writing nothing should produce nothing.
+        THROW_IF_FAILED(routines.FillConsoleOutputAttributeImpl(*screenInfo, red, 4, { 2, 1 }, cellsModified, false));
+        expected =
+            decsc() //
+            cup(2, 3) sgr_red("五六") //
+            decrc();
+        actual = readOutput();
+        VERIFY_ARE_EQUAL(4u, cellsModified);
+        VERIFY_ARE_EQUAL(expected, actual);
+    }
+
     TEST_METHOD(FillConsoleOutputCharacterW)
     {
         size_t cellsModified = 0;
@@ -407,7 +436,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
         actual = readOutput();
         VERIFY_ARE_EQUAL(expected, actual);
 
-        setupInitialContents();
+        setupInitialContents(false);
 
         // Writing at the start of a line.
         THROW_IF_FAILED(routines.FillConsoleOutputCharacterWImpl(*screenInfo, L'a', 3, { 0, 0 }, cellsModified, false));
@@ -453,7 +482,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
         std::string_view expected;
         std::string_view actual;
 
-        setupInitialContents();
+        setupInitialContents(false);
 
         // Scrolling from nowhere to somewhere are no-ops and should not emit anything.
         THROW_IF_FAILED(routines.ScrollConsoleScreenBufferWImpl(*screenInfo, { 0, 0, -1, -1 }, {}, std::nullopt, L' ', 0, false));
@@ -487,7 +516,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
         //
         //   m   n   M   N   o   p   O   P
         //
-        setupInitialContents();
+        setupInitialContents(false);
 
         // Scrolling from somewhere to somewhere.
         //
@@ -626,7 +655,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
         std::string_view expected;
         std::string_view actual;
 
-        setupInitialContents();
+        setupInitialContents(false);
 
         // Scrolling from nowhere to somewhere are no-ops and should not emit anything.
         THROW_IF_FAILED(routines.ScrollConsoleScreenBufferWImpl(*screenInfo, { 0, 0, -1, -1 }, {}, std::nullopt, L' ', 0, false));
@@ -660,7 +689,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
         //
         //   m   n   M   N   o   p   O   P
         //
-        setupInitialContents();
+        setupInitialContents(false);
 
         // Scrolling from somewhere to somewhere.
         //
@@ -797,7 +826,7 @@ class ::Microsoft::Console::VirtualTerminal::VtIoTests
             &screenInfoAlt));
 
         routines.SetConsoleActiveScreenBufferImpl(*screenInfoAlt);
-        setupInitialContents();
+        setupInitialContents(false);
         THROW_IF_FAILED(routines.SetConsoleOutputModeImpl(*screenInfoAlt, ENABLE_PROCESSED_OUTPUT | ENABLE_WRAP_AT_EOL_OUTPUT | ENABLE_VIRTUAL_TERMINAL_PROCESSING));
         readOutput();
 


Test patch: