Repository: terminal
Issue: ConPTY: transmit DECSET/DECRST state when client application enters/exits ENABLE_VIRTUAL_TERMINAL_INPUT mode
Labels: Product-Conpty,Area-Input,Issue-Bug,Priority-2

Certain applications (mainly those using the Cygwin/MSYS runtime) will attempt to change VT input parameters while VT input is technically off. Because we do nothing to communicate the expected VT input parameters to a connected terminal before we _turn on VT input passthrough_, it's possible for the connected terminal to disagree with the conhost acting as its pty host.

When the application enters `ENABLE_VIRTUAL_TERMINAL_INPUT` mode, ConPTY should send a string of `DECSET`/`DECRST` sequences to bring the connected terminal in line with it to keep up the illusion of passthrough.

Notes from #6737:

> From the Terminal's perspective, `ENABLE_VIRTUAL_TERMINAL_INPUT` is always on. It can't be lost/forgotten, because the terminal only generates VT. In general, it always has conhost sitting between it and the application to do translation from VT input. There are cases when the conhost sitting in the middle enters passthrough mode and dumps input straight out of terminal into the application.
> Therefore, Terminal needs to be aware of DECCKM. Whether it receives `DECCKM` and therefore whether it _generates the right cursor keys_, however, is based on whether the application has requested `ENABLE_VIRTUAL_TERMINAL_INPUT`.
> (There are applications that ask for DECCKM or other VT input format changes but are _not in VT input mode_, and sending them that data will freak them out.)
> 
> For WT 1.0, here's the matrix of behaviors.
> 
> <table><tbody>
> <tr><th>Action</th><th>VT Output Mode</th><th>VT Input Mode</th><th>Notes</th></tr>
> <tr>
>    <td rowspan="4">Application prints <code>\e[?1h</code></td>
>    <td>❌</td>
>    <td>❌</td>
>    <td rowspan="2">VT output is off, request ignored</td>
> </tr>
> <tr>
>    <td>❌</td>
>    <td>✅</td>
>    <!-- rowspanned -->
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>❌</td>
>    <td>conhost (PTY) (only) enables DECCKM</td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>✅</td>
>    <td>conhost (PTY) _and Terminal_ enable DECCKM</td>
> </tr>
> </tbody></table>
> <table><tbody>
> <tr><th>Action</th><th>VT Output Mode</th><th>VT Input Mode</th><th>PTY in DECCKM mode</th><th>Terminal in DECCKM mode</th><th>Notes</th></tr>
> <tr>
>    <td rowspan="8">User presses arrow key</td>
>    <td>❌</td>
>    <td>❌</td>
>    <td>❌</td>
>    <td>❌</td>
>    <td>Terminal emits <code>\e[A</code> or <code>\eOA</code><br>conhost (PTY) translates it into KEY_EVENT_RECORD</td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>❌</td>
>    <td>✅</td>
>    <td>❌</td>
>    <td>Terminal emits <code>\e[A</code><br>conhost (PTY) translates it into KEY_EVENT_RECORD</td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>❌</td>
>    <td>✅</td>
>    <td>✅</td>
>    <td>Terminal emits <code>\eOA</code><br>conhost (PTY) translates it into KEY_EVENT_RECORD</td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>✅</td>
>    <td rowspan="4"><em>passthrough mode</em></td>
>    <td>❌</td>
>    <td rowspan="2">Terminal emits <code>\e[A</code><br>conhost (PTY) <b>passes it through unmodified</b></td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>❌</td>
>    <td>✅</td>
>    <!-- rowspanned -->
>    <td>❌</td>
>    <!-- rowspanned -->
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>✅</td>
>    <!-- rowspanned -->
>    <td>✅</td>
>    <td rowspan="2">Terminal emits <code>\eOA</code><br>conhost (PTY) <b>passes it through unmodified</b></td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>❌</td>
>    <td>✅</td>
>    <!-- rowspanned -->
>    <td>✅</td>
>    <!-- rowspanned -->
> </tr>
> </tbody></table>
> 
> <details>
> <summary>BEHAVIORS BEFORE TERMINAL 1.0</summary>
> 
> <table><tbody>
> <tr><th>Action</th><th>VT Output Mode</th><th>VT Input Mode</th><th>Notes</th></tr>
> <tr>
>    <td rowspan="4">Application prints <code>\e[?1h</code></td>
>    <td>❌</td>
>    <td>❌</td>
>    <td rowspan="2">VT output is off, request ignored</td>
> </tr>
> <tr>
>    <td>❌</td>
>    <td>✅</td>
>    <!-- rowspanned -->
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>❌</td>
>    <td rowspan="2">conhost (PTY) (only) enables DECCKM</td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>✅</td>
>    <!-- rowspanned -->
> </tr>
> </tbody></table>
> <table><tbody>
> <tr><th>Action</th><th>VT Output Mode</th><th>VT Input Mode</th><th>PTY in DECCKM mode</th><th>Notes</th></tr>
> <tr>
>    <td rowspan="8">User presses arrow key</td>
>    <td>❌</td>
>    <td>❌</td>
>    <td rowspan="2"><em>ignored</em></td>
>    <td rowspan="2">Terminal emits <code>\e[A</code><br>conhost (PTY) translates it into KEY_EVENT_RECORD</td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>❌</td>
>    <!-- rowspanned -->
>    <!-- rowspanned -->
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>❌</td>
>    <td>✅</td>
>    <td>❌</td>
>    <td rowspan="2">Terminal emits <code>\e[A</code><br>conhost (PTY) translates it into <code>\e[A</code></td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>✅</td>
>    <td>❌</td>
>    <!-- rowspanned -->
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>❌</td>
>    <td>✅</td>
>    <td>✅</td>
>    <td rowspan="2">Terminal emits <code>\e[A</code><br>conhost (PTY) translates it into <code>\e[OA</code></td>
> </tr>
> <tr>
>    <!--<td rowspan="1">Application prints `\e[?1h`</td>-->
>    <td>✅</td>
>    <td>✅</td>
>    <td>✅</td>
>    <!-- rowspanned -->
> </tr>
> </tbody></table>
> 
> </details>

> _@DHowett_

> Okay, I'm literally seeing this behavior at the PTY:
> 
> * Input Mode = 0x9
> * Input Mode = 0x209
> * Input Mode = 0x9
> * Print out `DECCKM`
> * Input Mode = 0x209
> * Input Mode = 0x9
> 
> This happens with `printf`, this happens with less's startup (though it prints more than just DECCKM)`, this happens pretty much every time I launch a process.
> 
> It's quite literally disabling VT input mode _only while it is changing the input configuration_. If we'd ever made the console more strict ("VT input mode changes only apply when VT input mode is on"), this would break horribly. I wonder why they'd do this?
> _@DHowett_


> > Whether it receives `DECCKM` and therefore whether it _generates the right cursor keys_, however, is based on whether the application has requested `ENABLE_VIRTUAL_TERMINAL_INPUT`.
> 
> I totally did not expect that. If `ENABLE_VIRTUAL_TERMINAL_INPUT` is designed to affect which control sequences a pseudoconsole forwards from `WriteConsole` to the terminal, it is worth documenting.
> 
> > This regressed with #6485
> 
> Why doesn't Windows Terminal Preview 1.1.1812.0 have the same problem, then? [6ff8ba7](https://github.com/microsoft/terminal/commit/6ff8ba772890bf8abb8431295fe97c8e9e280dcb) was cherry-picked from [5e2c4c6](https://github.com/microsoft/terminal/commit/5e2c4c66e3dcb66c9d60769a2a028e39711168fd), which is included in that version.
> _@KalleOlaviNiemitalo_