Repository: flutter
Issue: Focus lost when interacting with keyboard
Labels: a: text input,framework,a: desktop,f: focus,has reproducible steps,P2,found in release: 3.7,found in release: 3.11,team-framework,triaged-framework

I was created the flutter widget with focus node and handle the keyboard(up, down, right, left arrow) interaction. but framework removes the focus when outside the widget have any interactional widget(button).

Eg., In the below sample have the custom widget for handle focus through keyboard and it is not working well in desktop platform

## Steps to Reproduce

1. Run the sample with below added code
2. Tap to request the focus to widget
3. Press multiple(3 to 4) times right arrow and press 2 times down arrow
4. Press up arrow to reproduce the issue


<details>
<summary>code sample</summary>

```dart

void main() {
  return runApp(
      MaterialApp(
    home: Scaffold(
      appBar: AppBar(
        title: Text('Demo'),
        leading: RawMaterialButton(
          child: Text('DemoButton'),
          onPressed: () {
            print('Pressed1');
          },
        ),
      ),
      body: CustomContainer(),
    ),
  )
      );
}

class CustomContainer extends StatefulWidget {
  @override
  _CustomContainerState createState() => _CustomContainerState();
}

class _CustomContainerState extends State<CustomContainer> {
  FocusNode _focusNode;
  int _selectedIndex = -1;

  @override
  void initState() {
    _focusNode = FocusNode();
    _selectedIndex = -1;
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return Container(
        child: GestureDetector(
            onTap: () {
              _focusNode.requestFocus();
            },
            child: RawKeyboardListener(
                focusNode: _focusNode,
                onKey: (RawKeyEvent event) {
                  if (event.runtimeType != RawKeyDownEvent) {
                    return;
                  }

                  if (_selectedIndex == -1) {
                    setState(() {
                      _selectedIndex = 0;
                    });
                  } else if (event.logicalKey ==
                      LogicalKeyboardKey.arrowRight) {
                    if ((_selectedIndex + 1) % 10 == 0) {
                      return;
                    }

                    setState(() {
                      _selectedIndex = _selectedIndex + 1;
                    });
                  } else if (event.logicalKey == LogicalKeyboardKey.arrowLeft) {
                    if (_selectedIndex % 10 == 0) {
                      return;
                    }

                    setState(() {
                      _selectedIndex = _selectedIndex - 1;
                    });
                  } else if (event.logicalKey == LogicalKeyboardKey.arrowUp) {
                    if (_selectedIndex - 10 < 0) {
                      return;
                    }

                    setState(() {
                      _selectedIndex = _selectedIndex - 10;
                    });
                  } else if (event.logicalKey == LogicalKeyboardKey.arrowDown) {
                    if (_selectedIndex + 10 >= 50) {
                      return;
                    }

                    setState(() {
                      _selectedIndex = _selectedIndex + 10;
                    });
                  }
                },
                child: GridView.count(
                  crossAxisCount: 10,
                  children: List.generate(50, (index) {
                    return Container(
                      height: 20,
                      color: _selectedIndex == index ? Colors.red : null,
                      child: Text(
                        'Item $index',
                        style: Theme.of(context).textTheme.headline5,
                      ),
                    );
                  }),
                ))));
  }
}
```
</details>



**Expected results:** 
- Widget focus not removed

**Actual results:**
- Widget focus removed

**Note:** This issue only occurred on desktop platforms(Mac Os, windows)

**Issue Replication Video:**

**Mac OS with app bar leading button issue replication:** 

https://user-images.githubusercontent.com/44917578/110765743-7ece4380-827a-11eb-8c87-de524ba9fa5c.mov

**Mac OS with out app bar leading button working fine:** 

https://user-images.githubusercontent.com/44917578/110765871-9dccd580-827a-11eb-9609-c747c7afc957.mov

**Web with app bar leading button working fine** 

https://user-images.githubusercontent.com/44917578/110765983-b5a45980-827a-11eb-890a-8cf4b6ad58ac.mov

<details>
  <summary>Flutter doctor</summary>

```
[✓] Flutter (Channel beta, 2.0.1, on Mac OS X 10.15.3 19D76 darwin-x64, locale en-US)                                                                                                                                                                                                    [!] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.
[!] Xcode - develop for iOS and macOS
    ! Xcode 11.3.1 out of date (12.0.1 is recommended).
      Download the latest version or update via the Mac App Store.
    ✗ Xcode requires additional components to be installed in order to run.
      Launch Xcode and install additional required components when prompted or run:
        sudo xcodebuild -runFirstLaunch
    ! CocoaPods 1.9.3 out of date (1.10.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.
[✓] Chrome - develop for the web
[✓] Android Studio (version 3.6)
[✓] VS Code (version 1.52.0)
[✓] Connected device (2 available)
```

<details>