Repository: microsoft/terminal (language: cpp)

Issue:
Japanese IME input mode switched back to Direct Input after pressing Enter key
### Windows Terminal version

1.14.1451.0

### Windows build number

10.0.19044.1766

### Other Software

_No response_

### Steps to reproduce

- Turn on Hiragana input mode
- Type something then press Enter to convert to japanese

### Expected Behavior

The input mode should not switch back to Direct Input

### Actual Behavior

The input mode switched to Direct Input automatically after conversion

![terminal](https://user-images.githubusercontent.com/76022206/176331241-3187170c-05e6-4abd-89bb-b3f9e74c9df6.gif)




Discussion / hints:
Might be related: https://github.com/microsoft/terminal/issues/12731
If we can't have a smarter way to detect when should we use compose mode and when should we use direct input, I would like to have an option to disable this behavior. Since I already installed a separate English (US) language along side Chinese (Simplified, MS Pinyin) so that I can turn off IME for good if I don't need it.
At least MS Pinyin handles it this way with conhost:

On first start of conhost, uses direct input by default.
If user enables compose mode with Shift, do not automatically switch back to direct mode.

I have encountered this issue, and it never occurred in previous versions.

When I write a git commit message mixed in English(alphabet) and Japanese, I often commit unintentional messages. (and have to amend commit again...)

Best regards.
> Might be related: #12731

It's indeed related as the problem occurred after upgrading to v1.14.1432.

Hope that there is a way to disable this behavior, because I tend to press enter key to end IME composition.
Automatically switch back to direct mode each time make it hard to input Japanese text in the terminal.

I also have the same problem.

When I type Chinese, then I press enter it jumps to English IME.
But I am not done with my Chinese typing.
This problem does not occur with [Google Japanese Input](https://www.google.co.jp/ime/).   
However, there is a bug that it automatically switches to "HankakuEisu" mode when starting Windows Terminal.

![20220724_121850](https://user-images.githubusercontent.com/13458509/180630761-b1600f84-d711-4f05-b84b-1ef8f484efff.gif)
I'm having the same problem with MS-IME on Windows Terminal version 1.14.1963.0.
I hope it fixes.  🙏 
When the Microsoft IME version was lowered, this event no longer occurred.
![image](https://user-images.githubusercontent.com/74097726/181139991-973fa1d1-b828-46f3-a8aa-8ce455fe2aef.png)


> When the Microsoft IME version was lowered, this event no longer occurred.

It didn't work for me. Does someone know other work-arounds?
If finalize with `Ctrl-m` instead of `Enter`, the IME will not turn off. But it is incredibly annoying method 😅

You could write following AutoHotKey script based on the hack.

```
#IfWinActive, ahk_exe WindowsTerminal.exe
  Enter::Send, ^{m}
#If
```
I'm running into the same issue.

In my environment, I can avoid the issue by waiting 0.5 - 1.0 seconds before pushing `enter`. Hope this helps some debugging.
I also have the same problem.
In my case, the workaround mentioned by @YossyHal works fine. (ver1.14.1963.0, on Windows11 22000.795)

> When the Microsoft IME version was lowered, this event no longer occurred.

ty.
I wonder if there's a way to only request AlphanumericHalfWidth the very first time the IME opens for a pane, and then just continue using the previously used mode for subsequent invocations. 

@YanceyChiew as an FYI. I'm not as familiar with the intricacies of the IME APIs, so you might have some better ideas here. 

Link for future me: #13028
> I wonder if there's a way to only request AlphanumericHalfWidth the very first time the IME opens for a pane, and then just continue using the previously used mode for subsequent invocations.

In short, yes, there is. 

This is the default behavior for Chinese input, so I thought there was no need for additional processing, but I didn't expect it to cause trouble for Japanese input method users.

---

**The root of the problem lies in [these](https://github.com/microsoft/terminal/blob/7976e48195a569c87e37b7ca4861dfdcb7db0d7c/src/cascadia/TerminalControl/TSFInputControl.cpp#L116-L129) few lines of code:**

```cpp
    void TSFInputControl::ClearBuffer()
    {
        if (!_inputBuffer.empty())
        {
            TextBlock().Text(L"");
            const auto bufLen = ::base::ClampedNumeric<int32_t>(_inputBuffer.length());
            _inputBuffer.clear();
            _editContext.NotifyFocusLeave();
            _editContext.NotifyTextChanged({ 0, bufLen }, 0, { 0, 0 });
            _editContext.NotifyFocusEnter();
            _activeTextStart = 0;
            _inComposition = false;
        }
    }
```

It can be seen that this code will not perform any effective operation when `_inputBuffer.empty()`, which is consistent with the performance of the Japanese input method that does not exit composition mode when press Enter directly with empty command line.

Maybe it is because the Microsoft Japanese input method is in the English (direct) input mode by default, and it's handling of lost input focus is also different from the Chinese input method, after each call to `_editContext.NotifyFocusLeave()`, it will exit from the composition input mode to the direct input mode. 

Incidentally, in another issue(#1304), someone wanted such a feature, but I found it difficult to implement cleanly.

 _(ps. When using the old Japanese input method, it will exit to direct input mode when pressing enter only when there are no characters in the input buffer of composition mode, so only newlines will cause composition mode to exit uncontrollably.)_

---

The easiest solution I can think of is to modify the `_editContext.InputScope` back to `CoreTextInputScope::Text` as soon as the input method gets focus for the first time. By testing in another example project, I found that it works fine with both Chinese and Japanese input methods. 
The key is that the timing of modification should be between the first and second calls to `_editContext.NotifyFocusEnter()`.

But in this project, I haven't found a suitable place to insert that line of code yet. It can also be inserted in the code block quoted above, `TermControl` will call it when Esc or Enter is pressed, inserting code anywhere before `_editContext.NotifyFocusLeave()` will take effect in time, but that looks a bit dirty doing pointless operations multiple times. 
It seems you are right.
I removed the changes in [PR #13028](https://github.com/microsoft/terminal/pull/13028/commits/fd324d54e953d70d54134b479d7833ce6de6b402) from the current HEAD(7976e4819) source, compiled it, and confirmed that it works fine with the current version of Japanese MS-IME.

I changed the line at at src/cascadia/TerminalControl/TSFInputControl.cpp:43

         _editContext.InputScope(CoreTextInputScope::AlphanumericHalfWidth);
to

         _editContext.InputScope(CoreTextInputScope::Text);

I agree with what you tried to do with your PR #13028,
we would like to find a good way to make both Japanese and Chinese users happy.

P.S.
However, there may be other causes as well.
It seems that this issue only occurs with the current version of Japanese MS-IME, not with Google-Input or MS-IME(previous ver). I also tried ATOK2022, a typical Japanese IME, but it didn't occur this issue.
> However, there may be other causes as well.
> It seems that this issue only occurs with the current version of Japanese MS-IME, not with Google-Input or MS-IME(previous ver). I also tried ATOK2022, a typical Japanese IME, but it didn't occur this issue.

This may be because `InputScope` is not implemented by the two non-Microsoft input methods mentioned above, its enum values ​​are simply ignored. And in my test, the old version of Microsoft Japanese input method will also be affected when wrapping.
So, the question now is, use the simple solution to change the `_editContext.InputScope` back to `CoreTextInputScope::Text` as soon as the `TSFInputControl` is connected to the input method for the first time, or add a setting to the `_editContext.InputScope` as originally envisaged if it doesn't satisfy all users? 

Both methods can solve the current problem, If we choose the former, since it cannot be assumed that the window is initialized with input focus, the code may have to be inserted before `_editContext.NotifyFocusLeave()`, making it itself or the code that determines whether it needs to be executed repeatedly. 

_Insert code [here](https://github.com/microsoft/terminal/blob/a9c3b7724c05a0da059c4a0433ea446f8e19cbff/src/cascadia/TerminalControl/TSFInputControl.cpp#L84-L91) and it will be repeated every time the window regains focus, not every time enter is pressed._
```cpp
    void TSFInputControl::NotifyFocusEnter()
    {
        if (_editContext != nullptr)
        {
            _editContext.NotifyFocusEnter();
            _focused = true;
           // CODE INSERT HERE
           // _editContext.InputScope(CoreTextInputScope::Text);
        }
    }

```

If we choose the latter, do we need to expose the option to the settings page in the gui, and have the default value determined by the user's system locale?
I think we should wait a while for other reports.
cf. @ironsand [says](https://github.com/microsoft/terminal/issues/13398#issuecomment-1198740633) that even using an older version of the IME did not work around this issue. 

I'd like to vote for "add a setting". And "Set default values determined by the user's system locale" is a good idea. With this, most people will not need to change this setting, so it seems to me that there is no problem even if editing the json is the only way to change this setting. (of course, it would be nice to have a gui)
I believe the easiest fix is to remove the calls to `NotifyFocusLeave()` and `NotifyFocusEnter()` inside `TSFInputControl::ClearBuffer()`. But I'm currently waiting to see whether I can fix the general behavior of the input first (#11479).

Fix patch:
diff --git a/.github/actions/spelling/expect/expect.txt b/.github/actions/spelling/expect/expect.txt
index 065cfd84a61..89da2c04b2a 100644
--- a/.github/actions/spelling/expect/expect.txt
+++ b/.github/actions/spelling/expect/expect.txt
@@ -42,7 +42,6 @@ antialias
 antialiasing
 ANull
 anycpu
-AOn
 APARTMENTTHREADED
 APCs
 api
@@ -80,7 +79,6 @@ ASingle
 asm
 asmv
 asmx
-AStomps
 ASYNCWINDOWPOS
 atch
 ATest
@@ -225,6 +223,7 @@ CFuzz
 cgscrn
 chafa
 changelist
+chaof
 charinfo
 charset
 CHARSETINFO
@@ -232,7 +231,6 @@ chcp
 checkbox
 checkboxes
 chh
-Childitem
 chk
 chrono
 CHT
@@ -2756,6 +2754,9 @@ xes
 xff
 XFile
 XFORM
+xin
+xinchaof
+xinxinchaof
 XManifest
 XMath
 XMFLOAT
diff --git a/src/cascadia/TerminalControl/TSFInputControl.cpp b/src/cascadia/TerminalControl/TSFInputControl.cpp
index 8cee7ffe154..7e9de19e99a 100644
--- a/src/cascadia/TerminalControl/TSFInputControl.cpp
+++ b/src/cascadia/TerminalControl/TSFInputControl.cpp
@@ -102,9 +102,7 @@ namespace winrt::Microsoft::Terminal::Control::implementation
             _inputBuffer.clear();
             _selection = {};
             _activeTextStart = 0;
-            _editContext.NotifyFocusLeave();
             _editContext.NotifyTextChanged({ 0, INT32_MAX }, 0, _selection);
-            _editContext.NotifyFocusEnter();
             TextBlock().Text({});
         }
     }
@@ -375,7 +373,7 @@ namespace winrt::Microsoft::Terminal::Control::implementation
                 incomingText);
             _selection = args.NewSelection();
             // GH#5054: Pressing backspace might move the caret before the _activeTextStart.
-            _activeTextStart = ::base::ClampMin(_activeTextStart, ::base::ClampedNumeric<size_t>(range.StartCaretPosition));
+            _activeTextStart = std::min(_activeTextStart, _inputBuffer.size());
 
             // Emojis/Kaomojis/Symbols chosen through the IME without starting composition
             // will be sent straight through to the terminal.
diff --git a/src/cascadia/TerminalControl/TermControl.cpp b/src/cascadia/TerminalControl/TermControl.cpp
index 943190cfaf2..6775c61e273 100644
--- a/src/cascadia/TerminalControl/TermControl.cpp
+++ b/src/cascadia/TerminalControl/TermControl.cpp
@@ -921,6 +921,19 @@ namespace winrt::Microsoft::Terminal::Control::implementation
             return;
         }
 
+        // GH#11479: TSF wants to be notified of any character input via ICoreTextEditContext::NotifyTextChanged().
+        // TSF is built and tested around the idea that you inform it of any text changes that happen
+        // when it doesn't currently compose characters. For instance writing "xin chaof" with the
+        // Vietnamese IME should produce "xin chào". After writing "xin" it'll emit a composition
+        // completion event and we'll write "xin" to the shell. It now has no input focus and won't know
+        // about the whitespace. If you then write "chaof", it'll emit another composition completion
+        // event for "xinchaof" and the resulting output in the shell will finally read "xinxinchaof".
+        // A composition completion event technically doesn't mean that the completed text is now
+        // immutable after all. We could (and probably should) inform TSF of any input changes,
+        // but we technically aren't a text input field. The immediate solution was
+        // to simply force TSF to clear its text whenever we have input focus.
+        TSFInputControl().ClearBuffer();
+
         _HidePointerCursorHandlers(*this, nullptr);
 
         const auto ch = e.Character();
@@ -1182,12 +1195,6 @@ namespace winrt::Microsoft::Terminal::Control::implementation
     {
         const auto window = CoreWindow::GetForCurrentThread();
 
-        if (vkey == VK_ESCAPE ||
-            vkey == VK_RETURN)
-        {
-            TSFInputControl().ClearBuffer();
-        }
-
         // If the terminal translated the key, mark the event as handled.
         // This will prevent the system from trying to get the character out
         // of it and sending us a CharacterReceived event.


Test patch: