Repository: flutter
Issue: [navigator2.0] Memory leak when remove page that not on the top of the stack
Labels: framework,f: routes,perf: memory,has reproducible steps,P2,team-framework,triaged-framework,found in release: 3.22,found in release: 3.24

### Steps to reproduce

1. build pages stack(a->b->c) use navigator pages api
2. go to page c, snapshot memory use devtools
3. at page c, add and remove page b many times
4. snapshot memory again, diff with step2

### Expected results

page b instance count is none or one

### Actual results

all page b instance retained by "Isolate → WidgetsFlutterBinding → BuildOwner " and not released

### Code sample

<details open><summary>Code sample</summary>

```dart
 @override
  Widget build(BuildContext context) {
    List<Page> pages = <Page>[
      MaterialPage(
        key: ValueKey('VeggiesListPage'),
        child: VeggiesListScreen(
          veggies: veggies,
          onTapped: _handleVeggieTapped,
        ),
      ),
      // this page leaks when add and remove repeatedly
      if (show404) MaterialPage(key: ValueKey('UnknownPage'), child: UnknownScreen()),
      if (_selectedVeggie != null)
        MaterialPage(
          key: ValueKey(_selectedVeggie!),
          child: VeggieDetailsScreen(
            veggie: _selectedVeggie!,
            show: show404,
            onTapped: () {
              show404 = !show404;
              notifyListeners();
            },
          ),
        ),
    ];

    return Navigator(
      key: navigatorKey,
      pages: pages,
      onPopPage: (route, result) {
        if (!route.didPop(result)) {
          return false;
        }

        // Update the list of pages by setting _selectedVeggie to null
        _selectedVeggie = null;
        show404 = false;
        notifyListeners();

        return true;
      },
    );
  }
```
[flutter_navigator_v2-master.zip](https://github.com/user-attachments/files/16244074/flutter_navigator_v2-master.zip)

</details>


### Screenshots or Video

<details open>
<summary>Screenshots / Video demonstration</summary>

[Upload media here]

https://github.com/user-attachments/assets/c8058a8e-cd3e-4a48-b4a2-c03d3ff7ee64

 
<img width="1662" alt="image" src="https://github.com/user-attachments/assets/ff813ab9-27c4-47e9-8815-0427df576b11">

</details>


### Logs

<details open><summary>Logs</summary>

leaks detail:

[snapshot_diff.csv](https://github.com/user-attachments/files/16244492/snapshot_diff.csv)

</details>


### Flutter Doctor output

<details open><summary>Doctor output</summary>

```console
flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.22.2, on macOS 14.4.1 23E224 darwin-x64, locale zh-Hans-CN)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
[✓] Xcode - develop for iOS and macOS (Xcode 15.3)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.3)
[✓] VS Code (version 1.91.0)
[✓] Connected device (3 available)
    ! Error: Browsing on the local area network for iPhone (205). Ensure the device is unlocked and attached with a cable or associated with the
      same local area network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)
    ! Error: Browsing on the local area network for iPhone. Ensure the device is unlocked and attached with a cable or associated with the same
      local area network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)
    ! Error: Browsing on the local area network for zhu的iPad. Ensure the device is unlocked and attached with a cable or associated with the
      same local area network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)
    ! Error: Browsing on the local area network for lin he的iPhone. Ensure the device is unlocked and attached with a cable or associated with
      the same local area network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)
[✓] Network resources

• No issues found!
```

</details>