Repository: flutter
Issue: Flutter needs a way to test real IME text input behavior
Labels: a: text input,framework,customer: crowd,c: proposal,P2,team-framework,triaged-framework

Super Editor is a Flutter-based document editing toolkit. Text manipulation is fundamental to everything we do. Specifically, we rely on text editing deltas, which are the standard tool for any non-trivial text editing use-cases.

Flutter lacks any mechanism to test real IME delta behavior. We've discovered that it's critical for us to be able to test real IME deltas for content changes.

Super Editor users are routinely filing bugs against our text manipulation behavior. These bugs are nearly impossible for us to diagnose and test against, given the current state of Flutter's IME system, including testing tools.

IME deltas are determined by a series of details:

1. User behavior with the software keyboard
2. The keyboard's decision about how to represent a change with deltas
3. Flutter's decision about how to organize and/or alter the deltas that the keyboard reports to it

Flutter lacks integration testing tools to simulate user behavior with the keyboard, and therefore lacks the ability to verify behavior for the above decision points. Instead, we're stuck writing tests that look like the following:

```dart
// Place caret at the start of the document.
      await tester.placeCaretInParagraph('1', 0);

      // Send initial delta, insertion of 'Goi'.
      await tester.ime.sendDeltas(
        const [
          TextEditingDeltaNonTextUpdate(
            oldText: '. ',
            selection: TextSelection.collapsed(offset: 2),
            composing: TextRange(start: -1, end: -1),
          ),
          TextEditingDeltaInsertion(
            oldText: '. ',
            textInserted: 'Goi',
            insertionOffset: 2,
            selection: TextSelection.collapsed(offset: 5),
            composing: TextRange(start: 2, end: 5),
          )
        ],
        getter: imeClientGetter,
      );

      // Simulate the auto-correction kicking in during the insertion of a '.'.
      await tester.ime.sendDeltas(
        const [
          // This delta represents the '.' typed by the user.
          TextEditingDeltaInsertion(
            oldText: 'Goi',
            textInserted: '.',
            insertionOffset: 3,
            selection: TextSelection.collapsed(offset: 4),
            composing: TextRange(start: -1, end: -1),
          ),
          // Deltas generated by the auto-correction.
          // First, delete everything.
          TextEditingDeltaDeletion(
            oldText: 'Goi.',
            deletedRange: TextRange(start: 0, end: 4),
            selection: TextSelection.collapsed(offset: 0),
            composing: TextRange(start: -1, end: -1),
          ),
          // Insert the auto-corrected word.
          TextEditingDeltaInsertion(
            oldText: '',
            textInserted: 'Going',
            insertionOffset: 0,
            selection: TextSelection.collapsed(offset: 5),
            composing: TextRange(start: -1, end: -1),
          ),
          // Insert the '.' typed.
          TextEditingDeltaInsertion(
            oldText: 'Going',
            textInserted: '.',
            insertionOffset: 5,
            selection: TextSelection.collapsed(offset: 6),
            composing: TextRange(start: -1, end: -1),
          ),
        ],
        getter: imeClientGetter,
      );

      // Ensure the text was inserted.
      expect(
        SuperEditorInspector.findTextInParagraph('1').text,
        'Going.',
      );
```

How did we accumulate those specific deltas? By manually running our app on a specific physical device and logging the deltas as they come through. Of course, even this approach misses what the OS actually reports. We only know what Flutter reports to our app. So this approach is both logistically impractical, and informationally incomplete.

But the point is that this test doesn't exercise any real keyboard, real IME, or even Flutter's own delta re-organization logic.

We'd like to write integration tests that interact with real iPhone, iPad, and Android software keyboards. We need to trigger the full round trip so that we can have some level of confidence that end-users will be able to edit text the way we expect.

We'd like to see something in Flutter akin to:

```dart
await tester.typeOnSoftwareKeyboard("Hello, world!");
```

No doubt there are some challenges here. Flutter would need to talk to the per-platform testing infrastructure to get to the keyboard. Also, especially on Android, there can be different keyboard implementations.

But pretty much any capability here is better than no capability at all. Even if we can only test against the standard iOS and Android keyboards, that would still go a long way to help us generate confidence about what end-users can successfully edit with our package.